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ABSTRACT 


The  Military  Traffic  Management  Command  (MTMC)  is  currently 
reengineering  the  DOD  Personal  Property  Shipment  and  Storage  Program.  DOD 
is  conducting  three  pilot  projects  in  support  of  this  effort.  Each  pilot  project 
represents  a  policy  alternative  for  improving  moving  services  for  military  families. 
The  Deputy  Under  Secretary  of  Defense  (Logistics)  tasked  United  States 
Transportation  Command  (USTRANSCOM)  to  evaluate  the  personal  property 
pilot  programs  as  part  of  Management  Reform  Memorandum  #  6.  This  thesis 
evaluates  the  policy  alternatives  for  reengineering  the  DOD  personal  property 
program  using  a  stakeholder  approach. 

The  study  develops  a  model  and  establishes  criteria  for  evaluating  the  three 
policy  alternatives.  Values  are  determined  for  the  criteria  by  interviewing  a 
sample  of  stakeholders  from  business,  government,  and  customers.  Policy 
alternatives  are  then  analyzed  from  each  stakeholder  perspective.  Finally,  a  policy 
alternative  is  identified  that  best  satisfies  the  criteria  for  each  stakeholder  as  well 
as  the  aggregate  of  stakeholders. 
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I.  INTRODUCTION 


A.  PERSONAL  PROPERTY  SHIPMENT  AND  STORAGE  PROGRAM 

The  Department  of  Defense  (DoD)  Personal  Property  Shipment  and  Storage 
Program  is  a  $1  billion  dollar  program  managed  for  the  military  services  by  the  Military 
Traffic  Management  Command  (MTMC).  DoD  is  the  moving  and  storage  industry’s 
single  largest  customer,  Avith  over  650,000  shipments  a  year  --  approximately  15 
percent  of  the  moving  industry  business  (Moore,  1995).  The  servicemember’s  quality 
of  life  is  effected  over  650,000  times  a  year  by  the  policies  that  govern  the  movement 
and  storage  of  their  possessions.  Over  a  military  career,  a  servicemember  is  typically 
required  to  move  more  than  20  times  (Moore,  1995). 

1.  Background  of  Reengineering  Effort 

The  DoD’s  Quality  of  Life  Task  Force,  established  by  Secretary  of  Defense 
William  J.  Perry  and  under  the  leadership  of  Mr.  John  O.  Marsh,  a  former  secretary  of 
the  Army,  mandated  that  systems  be  simplified,  the  quality  of  service  for  the 
servicemember  be  improved  and,  most  of  all,  that  members  be  satisfied  customers  of 
DoD’s  services.  The  National  Performance  Review  and  the  Acquisition  Reform 
Initiative  have  also  recognized  the  need  for  DoD  to  improve  the  processes  used  to 
regulate  and  manage  commercial  programs. 

Secretary  of  Defense  (SECDEF)  William  S.  Cohen,  in  his  Defense  Reform 
Initiative  Report,  November  1997,  stated  that  the  Department  must  do  a  better  job  of 
moving  DoD  servicemembers  and  their  families  to  their  new  duty  stations.  The  report 
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states  in  part,  .  .despite  the  fact  that  DoD  moves  more  household  effects  than  any 
United  States  corporation,  the  system  we  have  created  to  do  the  moving  has  given  our 
personnel  some  of  the  worst  service  in  the  nation  (Cohen,  1997),”  Improving  the 
current  personal  property  program  remains  a  critical  quality  of  life  issue,  and  is  listed  as 
one  of  the  SECDEF’s  nine  Defense  Reform  Initiatives. 

Under  the  current  program,  many  shipments  are  delayed,  lost,  stranded,  burned, 
and  damaged  (Moore,  1995).  Servicemembers  and  their  families  consistently 
experience  poor  service  from  movers,  excessive  incidence  of  loss  or  damage  to  their 
property,  and  a  laborious  claims  process  (GAO/NSIAD-97-49,  1996,  p.  1),  The  failures 
of  the  current  program  have  caused  hardships  throughout  the  military  for 
servicemembers  and  their  families. 

MTMC  is  currently  reengineering  the  program  to  improve  the  quality  of  these 
moves  and  reduce  costs,  while  attempting  to  not  adversely  impact  industry.  Efforts  to 
reengineer  the  personal  property  program  have  involved  numerous  governmental  and 
industry  organizations  including  Congress,  the  Secretary  of  Defense,  the  Military 
Services,  industry  trade  associations,  and  individual  transportation  service  providers. 
Though  MTMC  has  oversight  of  the  reengineering  pilots  under  the  cognizance  of  the 
United  States  Transportation  Command  (USTRANSCOM),  MTMC,  the  Army  and  the 
Navy  are  each  responsible  for  conducting  separate  pilots  to  evaluate  policy  alternatives. 
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2.  Organization  of  the  Personal  Property  Program 

MTMC  is  a  jointly  staffed,  major  Army  command  under  USTRANSCOM. 

While  MTMC  is  responsible  for  the  overall  operation  of  the  Personal  Property 
Shipment  and  Storage  Program,  each  Service  runs  offices  that  provide  customer  support 
to  servicemembers  in  the  process  of  moving.  MTMC  serves  as  the  single  contracting 
authority  for  personal  property. 

Although  the  DoD’s  domestic  personal  property  program  functions  under 
centralized  MTMC  management,  operations  are  carried  out  at  153  Military  Service- 
managed  Personal  Property  Shipping  Offices  (PPSOs)  (Beyer  and  Schwartz,  1993,  p.  1- 
2)}  Approximately  1,100  commercial  carriers  move  the  household  goods  of  military 
members  throughout  the  United  States  each  year. 

Every  six  months,  MTMC  solicits  bids  from  carriers  for  specific  domestic  traffic 
lanes  (routes).  In  response,  MTMC  receives  more  than  1.1  million  bids  for  domestic 
movements  fi’om  commercial  carriers.  Rates,  based  on  poundage,  mileage,  packing, 
labor,  and  storage,  are  established  and  proffered  to  moving  and  storage  companies  by  a 
MTMC  contracting  officer. 

A  successful  bid  to  MTMC’s  solicitation  places  the  contractor  on  a  local  area 
listing.  A  rotational  system  is  used  by  PPSOs  to  equitably  distribute  business  among 
accepted  carriers.  Every  domestic  PPSO  maintains  a  Traffic  Distribution  Record  (TDR) 


*  The  153  PPSOs  in  the  United  States  consist  of  45  Army,  30  Navy,  609  Air  Forces,  and  9  Marine  Corps.  The 
Coast  Guard  also  operates  21  PPSOs. 
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on  which  it  records  shipment  awards.  The  TDR  lists  carriers  according  to  their  filed 
rates,  with  the  lowest  rate  carriers  listed  first. 

The  moving  industry  contains  van  lines  with  agent  networks,  independent 
carriers,  agents,  and  forwarders.  Six  major  carriers  dominate  the  moving  industry.  In 
1996,  these  carriers  accounted  for  83  percent  of  the  market.  Many  small  businesses  in 
the  moving  industry  are  agents  for  one  of  the  major  carriers.  Many  carriers  create 
“paper  companies”  to  bid  and  gain  a  higher  percentage  of  the  traffic  allocation  while 
the  parent  company  conducts  the  move. 

B.  DESCRIPTION  OF  PILOT  PROJECTS 

Since  1994,  DoD  has  been  engaged  in  initiatives  to  reengineer  the  personal 
property  program.  On  June  21,  1994,  the  Deputy  Commander-in-Chief, 
USTRANSCOM,  directed  the  Army  component  of  USTRANSCOM,  the  Military 
Traffic  Management  Command  and  program  manager  for  DoD’s  Personal  Property 
Shipment  and  Storage  Program,  to  reengineer  the  personal  property  program 
(GAO/NSAIAD-97-49,  p.  30).  MTMC’s  initial  efforts  met  with  strong  industry 
resistance  and  hei^tened  congressional  interest.  One  June  15, 1995,  the  House 
Committee  on  National  Security,  concurred  that  DoD  must  pursue  a  higher  level  of 
service,  and  directed  that  DoD  undertake  a  pilot  program  to  implement  commercial 
business  practices  and  standards  of  service. 

Based  on  congressional  language  in  the  Fiscal  Year  1996  National  Defense 
Authorization  Act,  DoD  eventually  developed  three  pilot  programs  to  reengineer  the 
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Household  Goods  movement  proeess  (Personal  Property  Program  Management  Reform 
Memorandum  #6,  1999).  These  pilots  are  1)  the  Military  Traffic  Management 
Command  pilot;  2)  the  Navy’s  Servicemember  Arranged  Moves  (SAM)  pilot;  and  3) 
the  Army’s  Hunter  Airfield  pilot  which  was  later  expanded  into  the  Full  Service 
Moving  Project  (FSMP).  Figure  1  provides  an  organizational  overview  of  the  pilot 
project  hierarchy. 


Figure  1.  Pilot  Project  Hierarchy 


1.  MTMC  Reengineering  Effort 

The  MTMC  project  is  attempting  to  reengineer  the  current  program  in  order  to 

improve  the  move  process.  MTMC’s  pilot  aims  to  significantly  improve  the  quality 

assurance  and  procurement  methods  of  the  personal  property  program  through  the  use 
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of  a  Federal  Acquisition  Regulation  (FAR)-based  “best  value”  contract.  The  key 
features  of  this  program  include  selecting  carriers  based  on  servicemember  satisfaction 
and  past  performance  rather  than  simply  price;  achieving  stronger  carrier  commitment 
with  long-term  contracts;  and  offering  foil  replacement  value  protection  and  direct 
claims  settlement. 

MTMC’s  final  contract  solicitation  for  this  pilot  was  released  14  March  1997. 
Although  MTMC  received  proposals  in  June  1997,  pilot  implementation  was  delayed 
due  to  strong  industry  resistance  and  Congressional  concerns.  Concerned  about  the 
impact  that  the  competition  system  and  any  nonstandard  commercial  business  practice 
requirements  might  have  on  small  businesses,  the  statement  of  managers  accompanying 
the  1997  DoD  Authorization  Act  directed  that  DoD  report  on  the  impact  of  the  pilot 
program  on  small  business  (Warren,  1999).  After  reviewing  the  defense  reports  on 
small  bu  siness  impact,  the  House  Committee  on  National  Security  was  still  concerned 
that  MTMC’s  pilot  program  did  not  satisfactorily  address  issues  raised  by  small  moving 
companies.  As  a  result.  Congress  directed  that  the  Secretary  of  Defense  establish  a 
working  group  of  military  and  industry  representatives  to  develop  an  alternative  pilot 
project  (Warren,  1999). 

Although  the  working  group  reached  a  consensus  on  many  issues,  including  a  set 
of  program  goals,  it  could  not  reach  agreement  on  the  approach  to  take  for  the  pilot  test 
(Warren,  1999).  Consequently,  the  two  sides  presented  separate  proposals.  In 
November  1996,  the  General  Accounting  Office  (GAO)  reported  that  MTMC’s 
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proposal  met  the  goals  of  reengineering  the  personal  property  program  to  a  greater 
extent  than  the  industry  plan  (GAO/NSIAD-97-49,  p.  4). 

The  MTMC  pilot  program  implementation,  delayed  by  numerous  bid  protests, 
finally  commenced  in  January  1999.  The  pilot  reengineers  the  existing  program  for  50 
percent  of  the  moves  originating  in  three  states:  North  Carolina,  South  Carolina,  and 
Florida.  The  pilot  runs  concurrently  with  the  existing  MTMC-managed  program  at  the 
installations  in  the  three  states  and  will  involve  approximately  18,500  annual  moves 
(Warren,  1999).  The  pilot  is  scheduled  to  run  for  one-year  with  two  one-year  option 
periods.  (BGen.  Privratsky  Information  Paper,  1999). 

2.  Sailor  Arrangement  Move  (SAM) 

The  Department  of  Navy  initiated  a  separate  pilot  effort  in  1997  to  test  the 
option  of  allowing  servicemembers  to  choose  their  own  carrier  as  well  as  coordinate 
their  move  with  that  carrier.  The  Servicemember  Arranged  Move  (SAM)  pilot  is  a  low 
volume  pilot  based  on  voluntary  participation  featuring  payment  for  services  by  an 
international  merchant’s  procurement  authorization  card  (IMP AC).  The  servicemember 
selects  a  mover  from  a  list  of  carriers  that  are  self-certified  as  small  businesses  and 
approved  by  MTMC.  Servicemembers  are  then  able  to  coordinate  the  move  directly 
with  the  mover. 

Commencing  in  January  1998,  the  option  is  offered  for  shipments  originating  in 
the  areas  surrounding  Puget  Sound,  Washington;  San  Diego,  California;  Norfolk, 
Virginia;  and  Groton,  Connecticut  (GAO/T-NSIAD-99-106,  1999,  p.  4).  Eligible 
participants  must  limit  shipments  to  at  least  3,000  pounds  and  to  costs  between  $2,500 
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and  $25,000  (Warren,  1999).  Furthermore,  this  pilot  excludes  shipments  from  non¬ 
temporary  storage  or  warehouses. 

The  pilot  is  not  intended  to  replace  the  existing  MTMC-managed  program  and  is 
limited  to  Navy  military  members  with  permanent  change  of  station  orders  from  the 
participating  site.  Attempts  to  expand  the  program  to  include  all  servicemembers  and 
DoD  civilians,  and  to  incorporate  additional  objectives  were  not  approved. 

3.  Full  Service  Move  Program  (FSMP) 

Separate  from  the  MTMC  pilot  program,  the  Department  of  the  Army  is 
attempting  to  determine  whether  the  commercial  business  practice  of  outsourcing  can 
alleviate  known  problem  areas.  The  pilot  was  initiated  in  February  1996  as  a  Quality  of 
Life  initiative  to  improve  the  relocation  process  and  to  test  commercial  business 
practices  in  a  military  environment  (Warren,  1999).  The  pilot  effects  a  total  of  1,400 
annual  moves  that  originate  at  the  Hunter  Army  Airfield,  Savannah,  Georgia. 

On  January  3 1, 1997,  the  Army  selected  PHH  Relocation  (now  called  Cendant 
Mobility)  as  the  contractor  for  the  pilot.  Cendant  Mobility  is  the  largest 
relocation/move  management  company  in  the  world.  In  July  1997,  Cendant  Mobility 
began  providing  relocation  services  at  Hunter  Army  Airfield.  These  services  consisted 
of  point-to-point  move  management,  personal  move  counseling  and  coordination,  24 
hour  in-transit  visibility  of  the  shipment,  direct  claims  settlement  with  the  service 
member,  full  replacement  value  insurance,  and  a  single  point  of  contact  for  the 
servicemember  (Warren,  1999).  Additional  relocation  management  services  were 
optional  to  soldiers  and  included  services  such  as  home  finding  for  buyers  and  renters. 
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mortgage  services,  marketing  services  for  home  selling,  and  property  management. 
Throughout  the  entire  relocation  process,  the  contractor’s  move  counselor  is  the 
soldier’s  single  point  of  contact,  from  origin  to  destination,  from  entitlements 
counseling  to  claims  settlement  (LGen.  Cobum,  1999). 

The  initial  Hunter  Test  was  for  one  year,  ending  30  June  1998.  At  the  end  of  the 
initial  test,  an  extension  option  was  exercised  for  another  year.  On  Febmary  12, 1999, 
DoD  stated  that  it  intended  to  expand  the  Himter  Test  into  the  Full  Service  Moving 
Project  (FSMP)  with  certain  modifications  based  on  earlier  lessons  learned  from  the 
Hunter  Test.  Moreover,  the  Assistant  Deputy  Under  Secretary  of  Defense, 
Transportation  Policy  took  over  sponsorship  of  the  pilot  from  the  Army.  When  fully 
implemented,  FSMP  will  involve  all  military  Services  as  well  as  the  Coast  Guard.  Test 
areas  in  consideration  are  Georgia,  North  Dakota,  and  the  National  Capital  Region, 
which  include  approximately  45,000  annual  moves.  The  project  will  allow  DoD  to  test 
and  evaluate  a  larger  move  volume  as  well  as  Service-unique  requirements  under  the 
commercial  business  practice  concept  (FSMP  Overview,  1999).  The  FSMP  is  expected 
to  commence  in  late  1999. 

4.  DoD  Pilot  Project  Evaluation  Process 

The  USTRANSCOM  is  tasked  by  the  Deputy  Under  Secretary  of  Defense 
(Logistics)  with  overseeing  all  personal  property  pilots,  ensuring  consistency  in 
evaluation  criteria  and  assessment,  evaluating  the  pilots  to  determine  which  pilots  or 
tests,  or  portions  thereof,  could  provide  better  long-term  results,  and  then 
recommending  the  follow-on  course  of  action  and  time  line  for  implementation 
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throughout  DoD  (Evaluation  Plan  for  Reengineered  DoD  Personal  Property  Pilot, 

SAM,  and  DoD  FSMP,  1999).  Presently,  USTRANSCOM  is  working  with  the  services 
to  develop  a  common  set  of  data  measures  to  evaluate  the  pilot  project’s  results 
(GAO/T-NSIAD-99-106,  p.  5). 

Evaluation  methodology  currently  concentrates  on  four  main  areas:  quality  of 
life,  costs,  impacts  on  small  business,  and  process  improvement  innovations  (LGen. 
Thompson,  1999).  Quality  of  life  is  the  most  heavily  weighted  of  the  four  factors  and 
will  be  measured  by  the  servicemember’s  direct  feedback  as  to  whether  he/she  would 
use  that  carrier  again  (LGen.  Thompson,  1999).  Total  cost  includes  direct  and  indirect 
costs.  Direct  costs  include  contracted  services  and  goods  such  as  transportation,  storage 
and  accessorial  services.  The  indirect  baseline  cost  includes  management  of  the 
personal  property  program,  quality  assurance,  financial  transaction  processing,  and 
additional  systems  operations  and  maintenance  costs.  The  third  factor  considered,  the 
impact  on  small  business,  will  be  measured  against  the  Small  Business  Act,  Public  Law 
855536  that  establishes  a  government-wide  goal  for  small  business  participation  at  not 
less  than  23  percent  revenue  (LGen.  Thompson,  1999).  The  USTRANSCOM 
evaluation  program  will  also  identify  unique  features  of  each  pilot  program  that 
substantially  improves  the  service  to  the  customer.  These  features  include,  but  are  not 
limited  to:  the  use  of  move  management  services,  electronic  data  interchange  capability 
and  payment  features. 
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Although  standard  data  will  be  collected  from  each  pilot,  the  specific  methods  of 
data  collection  may  differ  somewhat  from  program  to  program  (LGen.  Thompson, 
1999).  Furthermore,  data  will  be  collected  and  compiled  for  each  pilot  by  a  different 
organization.  The  initial  Hunter  pilot  used  Electronic  Data  Systems  (EDS)  to  evaluate 
servicemember  quality  of  life,  cost,  and  small  business  participation  as  compared  to  the 
current  household  goods  move  process.  The  FMSP  will  use  the  installation 
transportation  office  /  traffic  management  office  (TMO)  and  the  relocation  service 
contractor  to  collect  the  baseline  and  test  data.  This  data  will  be  compiled  and  validated 
by  the  Army  Audit  Agency.  The  SAM  program  will  have  the  naval  base  personal 
property  shipment  offices  compile  the  baseline  and  test  data.  This  data  will  be 
forwarded  to  and  validated  by  the  Naval  Audit  Service.  GAO  has  repeatedly 
recommended  that  the  Secretary  of  Defense  develop  a  comprehensive  strategy  to  test 
each  of  the  policy  alternatives  (GAO/NSIAD-99-138,  1999,  p.  8). 

Ultimately,  all  test  data  will  be  forwarded  to  USTRANSOM  for  evaluation.  A 
USTRANSCOM  contraetor  will  design  and  conduct  surveys,  develop  and  implement 
the  evaluation  plan,  and  prepare  an  integrated  recommendation  plan  for  a  new  DoD 
personal  property  program  (Evaluation  Support  for  DoD’s  Personal  Property 
Reengineering  Programs  Statement  of  Work,  1999).  An  integrated  product  process 
team  (IPPT),  composed  the  0-5/0-6  level  representatives  from  the  Office  of  the 
Secretary  of  Defense,  USTRANSCOM,  the  Services,  DoD  Inspector  General,  and 
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Audit  Agencies  will  analyze  the  comparative  test  data,  develop  options,  and  forward 
them  to  an  Executive  Working  Group  (EWG). 

The  EWG  will  consist  of  Flag/General  Officer  level  representatives  from 
USTRANSCOM,  the  Services  and  OSD  (LGen.  Thompson,  1999).  The  charter  of  the 
EWG  is  to  make  final  recommendations  for  a  new  DoD  personal  property  program  to 
Commander-in-Chief  (CINC),  USTRANSCOM  (LGen.  Thompson,  1999). 

CESIC,  USTRANSCOM  will  approve  or  make  modifications  to  EWG’s 
recommendation(s)  after  a  review.  As  the  DoD  Executive  Agent  for  the  evaluation  of 
all  DoD  personal  property  pilot  programs,  CINC,  USTRANSCOM  will  submit  a  final 
recommendation(s)  to  the  Secretary  of  Defense  by  December  2000. 

5.  Summary 

Since  1994,  DoD  has  pursued  initiatives  to  reengineer  the  personal  property 
program.  DoD  has  attempted  to  involve  industry  in  these  efforts,  but  there  has  been 
minimal  agreement  on  pilot  characteristics.  DoD  is  currently  testing  three  different 
approaches  to  reengineering  the  personal  property  program  under  separate  pilot 
projects.  Figure  2  summarizes  the  characteristics  of  these  pilots.  These  projects  effect 
approximately  21,000  annual  moves.  The  expansion  of  the  Hunter  pilot  into  the  FSMP 
will  increase  participation  in  the  pilots  to  approximately  66,000  annual  moves  or  10 
percent  of  total  annual  moves.^  Final  recommendations  will  be  forwarded  to  the 
Secretary  of  Defense  from  CINC,  USTRANSCOM  by  December  2000. 

^  The  personal  property  program  covers  approximately  650,000  annual  moves. 

12 


SAM 


Hunter  (FSMP 


MTMC 


•  FAR-based  contract-  • 

commercial  LOA  • 

•  Full  Value  Protection 

(Maximum  $72K)  • 

•  Direct  claims  settlement  w/ 

carrier  • 

•  Claim  settlement  w/in60  • 

•  Carriers  approved  and  in  good 
standing  with  HQMTMC 

•  Inclusion  of  the  Service  • 

Contract  Act 

•  Inconvenience  payments 

•  Customer  satisfaction  surveys 

•  Tailored  pre-move  counseling 
byPPSO  . 

•  Shipment  tracking  (ITV) 

establishment  of  a  toll  free  #  to 
be  manned  five  days  a  week 
(M-F)  from  0700  to  1800  • 

Central  Time 

•  Use  of  Pagers  to  support  direct 
delivery 

•  Pricing  strategy  based  on 

commercial  tariff  * 

•  Weight  restrictions  reduced  . 

from  3000  to  1000  lbs.  For  ^ 

(HHG) 

•  SIT  increased  from  90  days  to  ^ 

360  days  ^ 

•  Servicemember  chooses  carrier 
from  among  available 
participating  carriers 

•  Servicemember  may  interview  ^ 

and/or  visit  carrier  ^ 

•  Carriers  may  service  any  or  all 
channels 

•  1 00%  small  business 

•  Government  Credit  Card  ^ 

payment 


FAR-based  contract 

Full  Replacement  Protection 

(Maximum  $75K) 

Direct  claims  settlement  w/  move 
manager 

Claims  settlement  w/in  30  days 
The  Contractor  obtains  est.  for 
determining  the  replacement 
value/repair  cost  of  the  property 
Transportation  provider  qualifications: 

•  Financial  stability 

•  Ensured  capacity 

•  Past  performance  standards 

•  Loss/damage  prevention 
measures 

•  Guaranteed  on-time  pick-up  and 
delivery 

•  Inconvenience  payments 
Move  Manager  qualifications: 

•  Performance  risk  assessment 

•  Past  performance 

•  Ability  to  meet  SOW 
requirements 

Monthly  Move  Analysis  Report 

Pre-move  counseling 

Point  to  point  move  management 

Binding  Estimates  for  excess  costs 

Destination  information 

Shipment  tracking  via  a  toll-free  #  and 

an  electronic  address.  Service  shall  be 

available  24  hours  a  day,  7  days  a 

week 

Real  Estate  Services 
Domestic  pricing  based  on 
commercial  tariff 

International  pricing  based  on  Single 
Factor  Rate 

Transportation  providers  pay 
commissions  to  move  managers 
Distribution  based  on  Quality 
Use  of  move  management  service 

1  year  agreement  with  transportation 
providers 

2  year  contract  w/  3  option  years  with 
move  manager 

Contractor  required  to  meet  small 
business  goals  in  accordance  with  SBA 
Use  Bank  Service  to  pay 


•  FAR-based  contract 

•  Full  Replacement  Protection  (Maximum 
$63K) 

•  Claims  settlement  w/  Government 

•  The  Contractor  obtains  est.  for 
determining  the  replacement 
value/repair  cost  of  the  property 

•  Carrier  qualifications 

•  Committed  Daily  Capacity 

•  Past  performance  standards 

•  Established  quality  measures 

•  Personnel  requirements 

•  Carriers  evaluated  on  on-time 
pickup  &  delivery  and  damage 
rates 

•  Inconvenience  payments 

•  Use  of  Customer  Survey 

•  Quarterly  performance  reviews 

•  Claims  Activity  Report 

•  Pre-move  counseling 

•  Shipment  tracking  (ITV)  toll  free  #  that 
is  operational  7  days  a  week,  24  hrs/day. 
Must  response  during  working  hours. 

•  Pricing  Strategy  based  on  commercial 
tariff 

•  Traffic  allocated  on  a  rotational 
capacity  to  the  contract  min. 
guaranties  meet  then  on  best  value 

•  EDI:  Transportation  Operational 
Personal  Properly  Standard  System 
(PTOPS) 

•  Payment  through  DFAS 

•  Compliance  with  SBA 


Figure  2.  Program  Comparison 
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C.  PURPOSE  OF  THESIS 


The  DoD  is  planning  or  conducting  three  pilot  projects  to  reengineer  its  personal 
property  shipping  program.  Each  pilot  project  represents  policy  alternatives  for 
improving  moving  services  for  military  families.  MTMC  is  responsible  for  evaluating 
all  of  the  pilots;  its  intention  is  to  determine  which  one  could  provide  better  long-term 
service  (GAO/NIAD-99-138,  pg.  2). 

D.  RESEARCH  QUESTIONS 

The  primary  research  question  of  this  thesis  is:  Which  policy  alternative  for 
reengineering  the  DoD’s  personal  property  program  best  satisfies  stakeholder 
criteria? 

Subsidiary  research  questions  supporting  this  issue  are: 

1 .  What  are  the  policy  alternatives  for  reengineering  the  personal  property 
program? 

2.  What  are  the  criteria  for  evaluating  for  policy  alternatives? 

3 .  Who  are  the  stakeholders  for  the  reengineering  effort? 

4.  What  value  do  the  stakeholders  place  on  the  evaluation  criteria? 

5.  Which  policy  alternative  fulfills  stakeholder  criteria? 

E.  SCOPE  AND  METHODOLOGY 

This  thesis  provides  background  on  the  DoD’s  effort  to  reengineer  the  personal 
property  program,  identifies  the  dominant  stakeholders  of  this  effort,  develops  a  single 
set  of  stakeholder  criteria,  and  evaluates  the  alternate  proposals  with  respect  to  these 
criteria. 
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This  effort  employs  the  basic  tools  of  evaluation  research.  Specifically,  the 
design  involves  collecting  individual  opinions  and  archival  data  using  primary  and 
secondary  sources  on  three  alternate  proposals  to  reengineer  the  personal  property 
program.  The  archival  data  include  the  study  of  government  and  industry  documents. 
Using  this  documentation,  a  single  set  of  criteria  is  developed  to  evaluate  the  policy 
alternatives. 

A  stakeholder  map  is  developed  to  identify  dominant  stakeholders.  Individuals 
within  each  dominant  stakeholder  group  are  chosen  using  a  convenience  sample  and 
interviewed  using  a  telephonic  questionnaire.  The  collected  data  are  coded  and 
introduced  into  a  spreadsheet.  Measures  of  central  tendency  are  calculated  for  each 
criterion.  Based  on  the  value  stakeholders’  place  on  the  criteria,  one  proposal  is 
identified  that  is  the  best  fit  for  each  stakeholder  group’s  expectations.  Furthermore, 
one  proposal  is  identified  that  is  the  best  fit  for  all  major  stakeholders. 

F.  LIMITATIONS 

Recommendations  are  limited  to  the  current  pilot  projects.  Stakeholder 
expectations  are  determined  in  terms  of  the  pilot  projects.  There  are  additional 
stakeholder  criteria  but  since  they  were  not  incorporated  into  the  policy  alternatives, 
they  were  not  included  in  this  study.  This  study  focuses  on  determining  which  policy 
altemative(s)  best  reflect  the  stakeholders’  concerns  and  meets  their  needs.  This  study 
does  not  attempt  to  address  problem  definition  nor  implementation  issues. 
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G.  ORGANIZATION  OF  STUDY 

Chapter  I,  Introduction,  presents  the  focus  of  this  study.  This  chapter  introduces 
the  effort  to  reengineer  DoD’s  personal  property  program,  describes  the  pilot  projects 
and  outlines  DoD’s  policy  evaluation  process.  Furthermore,  this  chapter  establishes  the 
research  question  and  methodology,  and  presents  an  outline  of  the  thesis. 

Chapter  II,  Literature  Review,  provides  a  definition  and  an  overview  of  policy 
analysis  in  the  public  sector.  An  analysis  process  is  also  identified  and  reviewed.  This 
chapter  will  provide  an  overview  of  the  stakeholder  concept.  A  method  for  conducting 
a  stakeholder  audit  is  described. 

Chapter  III,  Methodology,  articulates  the  research  methodology  used  to  conduct 
this  study.  A  goals  and  evaluation  criteria  map  is  presented.  The  extent  to  which  each 
pilot  project  fulfills  specified  criteria  is  determined.  Finally,  a  stakeholder  analysis  is 
conducted. 

Chapter  IV,  Data,  presents  summary  statements  and  data  garnered  from  the 
telephonic  interviews.  Measures  of  central  tendency  are  developed  for  each  stakeholder 
group  based  on  the  evaluation  criteria. 

Chapter  V,  Analysis,  discusses  and  analyzes  the  data  collected  in  the  telephonic 
interviews. 

Chapter  VI,  Conclusions  and  Recommendations,  provides  conclusions  that  can 
be  drawn  from  an  analysis  of  the  data.  Specifically,  this  chapter  will  recommend  the 
policy  altemative(s)  that  best  fulfills  stakeholder  expectations. 
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IL  LITERATURE  REVIEW 


A.  INTRODUCTION 

Public  policy  analysis  or  evaluation  determines  which  alternative  public 
policies  will  optimize  on  a  given  set  of  goals  (Nagel,  1990,  p.  ix).  Related 
concepts  include  policy  studies,  program  evaluation,  public  management  science, 
and  policy  science.  Policy  evaluation  and  analysis  emphasizes  examining 
alternative  public  policies,  as  contrasted  to  describing  them  or  explaining  why  they 
exist  (Nagel,  p.  x).  This  chapter  provides  an  overview  of  public  policy  analysis  in 
the  public  sector,  introduces  a  model  for  performing  policy  analysis,  and 
demonstrates  how  this  model  can  be  applied  to  the  research  question. 

B.  POLICY  ANALYSIS  IN  THE  PUBLIC  SECTOR 

Public  policy  analysis  is  a  systematic  approach  to  making  policy  choices  in 
the  public  sector  (Walker  and  Fisher,  1994,  p.  1).  Its  purpose  is  to  assist 
policymakers  in  choosing  a  preferred  course  of  action  amid  multiple  complex 
alternatives.  The  policy  analysis  process  gathers  and  presents  the  information  that 
assists  the  policymaker  in  making  a  decision  between  policy  alternatives. 

Much  of  the  growing  literature  in  the  area  of  public  policy  addresses 

various  approaches  to  policy  analysis.  Weimer  and  Vining  identify  three 

approaches:  (1)  the  objective  model,  (2)  the  client  advocate  model,  and  (3)  the 

issue  advocate  model  (Weimer  and  Vining,  1991,  p.  18).  This  is  similar  to  the 

approaches  described  by  Fischer,  Frank  and  Forester:  (1)  the  science  model,  (2) 
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the  counsel  model,  and  (3)  the  advocate  model  (Fischer,  Frank,  and  Forester, 

1987,  p.  138). 

The  objective  or  science  model  limits  the  analyst  to  questions  that  science 
can  properly  answer;  that  is,  to  questions  of  empirical  fact  and  practical  means 
(Fischer,  Frank,  and  Forester,  p.  137).  The  client  advocate  or  counsel  model 
focuses  on  policies  that  provide  the  greatest  benefit  to  their  client.  Finally,  many 
authors  endorse  the  issue  advocate  model.  This  approach  promotes  policies  that 
achieve  the  greatest  good  for  society  as  a  whole.  In  general,  the  analyst  need  not 
adopt  any  one  of  the  approaches  in  its  extreme  form.  Rather  than  selecting  one  of 
the  three  fundamental  approaches,  the  analyst  should  attempt  to  keep  all  three 
under  consideration  (Weimer  and  Vining,  p.  19). 

There  are  several  alternative  ways  to  analyze  policy  options.  They 
allcontain  similarities,  but  each  uses  different  key  words  to  describe  a  particular 
framework  and  each  goes  into  varying  degrees  of  detail  (Bonser,  McGregor,  and 
Oster,  1996,  p.  48).  The  policy  evaluation  process  generally  involves  performing 
the  same  set  of  logical  steps,  not  always  in  the  same  order.  Walker  and  Fisher 
present  the  following  model  as  seen  in  Figure  3.  This  model  is  very  similar  to  the 
process  laid  out  by  Stokey  and  Zeckhouser,  Mason  and  Mitroff,  and  by  Bonser, 
McGregor  and  Oster. 
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Figure  3.  Generic  Policy  Evaluation  Model 


C.  MODEL  FOR  EVALUATING  POLICY  ALTERNATIVES 

The  jSramework  shown  in  Figure  4  is  a  hybrid  drawn  from  several 
approaches.  The  available  literature  provides  the  basic  components  of  this  model. 
However,  this  model  places  a  different  emphasis  on  some  steps  and  integrates 
other  steps  to  provide  a  rational  and  logical  basis  for  evaluating  this  case. 
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1 .  state  the  problem 


_ i 

2.  Identify  solutions 


Figure  4.  Proposed  Policy  Evaluation  Model 


1.  State  the  Problem 

The  first  step  sets  the  boundaries  and  the  context  of  the  problem  to  be 
studied.  It  involves  identifying  the  questions  or  issues  involved,  and  the  desired 
end  state  or  goals.  Proposals  for  policy  changes  generally  arise  out  of  the 
dissatisfaction  of  one  or  more  groups  of  stakeholders  with  the  efficiency  or 
effectiveness  of  existing  policies  or  programs  (Rossi  and  Freeman,  1989,  p.  73). 
Understanding  the  problem  involves  investigating  the  symptoms  that  prompt 
stakeholder  dissatisfaction  (Weimer  and  Vining,  p.  212).  An  especially  critical 
consideration  in  the  explication  of  the  problem  is  identifying  the  gap  between  the 
current  policy  or  program  outputs  and  outcomes  and  the  desired  output  and 
outcomes. 
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Program  planning  is  predicated  on  a  need  to  reduce  the  gap  between  the 
desired  state  of  affairs  and  what  actually  exists—in  other  words  between  a  goal  and 
reality  (Rossi  and  Freeman,  p.  121).  Goals  are  generally  abstract  idealized 
statements  of  outcomes  (Rossi  and  Freeman,  p.  121).  Policy  evaluation  is  often 
complicated  by  the  existence  of  multiple  goals  that  are  not  clearly  defined.  Many 
programs,  especially  those  with  a  large  target  population  or  far-reaching  effects, 
initially  state  their  goals  in  broad  and  often  vague  terms.  Stakeholders  with 
varying  and  sometimes  conflicting  needs,  interests,  and  perspectives  define  the 
goals  of  a  program. 

2.  Identify  Solutions 

This  step  identifies  alternative  policies  that  provide  concrete  steps  to  solve 
the  stated  problem.  The  majority  of  literature  focuses  on  developing  alternative 
solutions.  These  authors  provide  a  sequence  of  activities  that  identify,  design,  and 
screen  possible  policy  alternatives.  However,  as  Weimer  and  Vining  point  out 
existing  policy  proposals  often  exist.  “There  are  really  four  sources  for 
developing  policy  alternatives:  (1)  existing  policy  proposals;  (2)  generic  policy 
solutions;  (3)  “modified”  generic  policy  solutions;  and  (4)  custom-made  solutions 
(Weimer  and  Vining,  p.  225).”  The  policy  analyst  must  be  careful  to  avoid 
starting  with  a  preconceived  idea  of  the  preferred  alternatives  even  when 
evaluating  existing  policy  proposals  (Bonser,  McGregor,  and  Oster,  p.  49). 
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Moreover,  the  policy  alternatives  should  be  mutually  exclusive.  “You 
should  ensure  that  your  alternatives  are  mutually  exclusive;  they  are,  after  all, 
alternative  policies  (Weimer  and  Vining,  p.  227).”  Alternatives  are  obviously  not 
mutually  exclusive  if  their  focus  or  objectives  are  similar.  In  such  circumstances, 
these  policies  may  not  provide  distinct  choices,  and  should  be  combined  or 
eliminated. 

3.  Develop  Evaluation  Criteria 

For  a  worthwhile  evaluation  to  be  undertaken,  goal  statements  must  be 
refined  and  stated  in  terms  that  can  be  measured,  that  is,  operationally  defined 
(Rossi  and  Freeman,  p.  121).  “For  evaluation  purposes,  goal  setting  must  lead  to 
the  operationalization  of  the  desired  outcome;  that  is  the  condition  to  be  dealt  with 
must  be  specified  in  detail,  together  with  one  or  more  criteria  (Freeman  and  Rossi, 
p.  121).”  Criteria  are  required  to  evaluate  the  extent  to  which  each  alternative 
policy  solves  the  stated  problem. 

This  step  is  especially  difficult  when  diverse  groups  have  agreed  on  broad 
goals  but  cannot  agree  on  the  objectives  or  sub-goals  that  support  those  goals.  In 
this  step  the  analyst  must  decide  on  the  criteria  with  which  to  evaluate  alternative 
policies  (Walker  and  Fisher,  p.  2).  “Loosely  speaking,  a  policy  is  a  set  of  actions 
taken  to  solve  a  problem  (Walker  and  Fisher,  p.  2).”  The  policymaker  has  certain 
criteria  that,  if  met,  would  “solve”  the  problem. 
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Identifying  these  criteria  is  a  critical  step  in  evaluating  competing  policies. 
When  multiple  competing  stakeholders  are  present,  it  is  necessary  to  create  a 
single  set  of  measurement  criteria  by  which  alternative  policies  may  be  compared. 
Therefore,  establishing  criteria  requires  either  assumptions  or  knowledge  about 
two  fundamental  aspects  of  the  situation:  policy  goals  and  stakeholder  values. 
Policy  goals  incorporate  the  set  of  actions  that  represent  a  solution  to  the  problem. 
Stakeholder  values  provide  a  means  to  evaluate  the  policy  alternatives  from  the 
perspective  of  each  stakeholder. 

4.  Evaluate  Solutions  Based  on  Criteria 

In  this  step,  the  alternative  policies  are  analyzed  and  compared  according  to 
established  criteria.  “When  the  analysis  of  several  alternatives  has  been 
completed,  the  policy  analyst  begins  to  synthesize  the  results  (Bonser,  McGregor, 
and  Oster,  p.  50).”  Each  alternative  must  be  ranked  according  to  the  criteria 
chosen  by  the  analyst  in  order  to  determine  the  alternative  that  is  most  likely  to 
fulfill  the  criteria  (Bonser,  McGregor,  and  Oster,  p.  50). 

5.  Recommend  the  Preferred  Alternative 

Finally,  the  evaluator  should  recommend  the  preferred  alternative. 
“Choosing  the  best  alternative  is  trivial  when  you  have  an  alternative  that  ranks 
highest  on  all  criteria  (Weimer  and  Vining,  p.  234).”  Unfortunately,  this  rarely 
occurs.  Rarely  is  the  best  policy  also  the  preferred  policy.  “Although  you  may 
sometimes  be  pleasantly  surprised,  you  should  expect  to  find  different  alternatives 
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doing  best  on  different  criteria  (Weimer  and  Vining,  p.  234).”  When  alternative 
policies  fulfill  the  established  criteria,  but  to  differing  degrees,  the  analyst  must 
make  a  determination  as  to  which  criteria  should  be  given  greater  weight.  This 
determination  is  often  subjective  and  based  on  the  evaluator’s  approach  or 
perspective.  In  this  situation  there  are  implicit  trade-offs  between  criteria.  The 
evaluator  must  be  explicit  in  stating  the  values  and  trade-offs  that  were  made  in 
determining  the  preferred  alternative. 

D.  APPLICATION  OF  MODEL  TO  STUDY 

The  policy  evaluation  model  developed  in  Figure  4  is  used  in  this  study  to 
determine  which  policy  alternative  for  reengineering  DoD’s  personal  property 
program  best  satisfies  stakeholder  criteria.  Figure  5  identifies  the  steps  in  the 
model  and  the  supporting  tasks.  Figure  5  also  provides  an  overview  of  each  step 
in  terms  of  the  chapter  location,  the  corresponding  figures,  the  responsible  party, 
and  the  procedure  used. 
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thodology  Overview 

1.  Identify  Problem 

Within  the  context  of  the  personal  property  program,  there  has  been  no 
single  and  certain  definition  of  program  characteristics  requiring  change.  While 
senior  government  officials  have  repeatedly  identified  shortcomings  in  the  current 
program,  criticism  has  focused  on  negative  outcomes,  not  on  specific  outputs  of 
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the  program.^  In  GAO/NSIAD  reports  90-50  (1990)  and  94-26  (1993),  GAO 
recommended  that  MTMC  replace  or  modify  the  domestic  household  goods 
bidding  system  and  that  DoD  adopt  commercial  practices,  such  as  using  a  smaller 
number  of  carriers.  These  problems,  a  high  rate  of  claims,  and  general  customer 
dissatisfaction  with  the  move  process  have  formed  the  basis  of  DoD  efforts  to 
reengineer  the  personal  property  program.  This  study  will  not  attempt  to  identify 
specific  issues  or  problems  inherent  in  the  personal  property  program. 

2.  Identify  Solutions 

As  previously  noted,  there  are  three  separate  alternative  solutions  to  the 
problems  inherent  to  the  personal  property  program.  These  polices  do  not 
necessarily  represent  the  best  set  of  alternatives,  but  other  analysts  have  found 
them  to  be  plausible  responses  to  the  policy  problems.  Furthermore,  the 
alternatives  are  mutually  exclusive  and  have  a  sufficiently  different  focus.  In  this 
study,  research  and  subsequent  analysis  will  focus  on  these  existing  policy 
proposals. 

3.  Develop  Criteria 

Establishing  criteria  requires  either  assumptions  or  knowledge  about  the 
situation:  policy  goals  and  stakeholder  values.  Numerous  government  and 
industry  documents  provide  an  initial  list  of  the  goals  of  the  various  alternatives. 

^  Note  the  previously  cited  comments  of  the  Secretary  of  Defense  and  the  congressional  testimonies  of  Mr. 
Roger  Kallock,  Deputy  Under  Secretary  of  Defense  (Logistics),  Lieutenant  Generals  Thompson  and 
Cobum,  and  Mr.  David  Warren,  Director  of  Defense  Management  Issues,  GAO. 


However,  it  is  necessary  to  determine  how  each  alternative  attempts  to  solve  the 
stated  problem.  In  other  words,  it  is  necessary  to  link  the  goals  of  the  personal 
property  reengineering  program  to  the  sub-goals  and  criteria  of  each  alternative. 
This  schematic  or  map  allows  the  evaluator  to  develop  a  set  of  criteria  that  can  be 
used  to  evaluate  the  extent  to  which  each  policy  alternative  meets  a  specific 
criterion.  Chapter  III,  Methodology,  details  the  criteria  development  process. 

4.  Evaluate  Policy  Alternatives 

Once  the  evaluation  criteria  are  established,  the  analyst  must  determine  the 
value  to  be  placed  on  each  criterion  when  evaluating  and  comparing  policy 
alternatives.  Value  is  based  on  perspective.  In  this  study,  the  perspective  of  each 
dominant  stakeholder  group  is  assessed.  Chapter  III,  Methodology,  describes  the 
steps  in  this  assessment.  Chapter  IV,  Data,  summarizes  the  values  stakeholders 
place  on  the  goals,  sub-goals  and  evaluation  criteria.  Chapter  V,  Data  Analysis, 
uses  the  evaluation  criteria  and  interview  results  to  evaluate  each  policy  alternative 
from  the  stakeholders’  perspectives. 

5.  Recommend  Preferred  Alternative 

The  policy  analysis  process  deals  with  complex  problems  and,  most 
importantly,  multiple  goals.  It  is  unlikely  that  any  policy  is  going  to  be  ideal  in 
terms  of  all  goals.  However,  by  establishing  a  single  set  of  evaluation  criteria  and 
determining  the  value  that  dominant  stakeholders  place  on  those  criteria,  it  is 
possible  to  recommend  a  preferred  alternative  from  a  particular  perspective.  More 
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importantly,  it  is  possible  to  establish  a  set  of  evaluation  criteria  based  on  the 
aggregate  values  of  stakeholders.  Chapter  VI,  Conclusions  and 
Recommendations,  identifies  the  policy  that  is  optimal  for  each  stakeholder  group 
as  well  an  alternative  that  is  in  the  best  interests  of  all. 

E.  CHAPTER  SUMMARY 

Policy  analysis  “is  not  a  specific  methodology,  but  it  makes  use  of  a  variety 
of  methodologies  in  the  context  of  a  generic  framework  (Walker  and  Fisher,  p. 

1).”  It  often  proves  difficult  to  establish  the  boundaries  and  context  of  the 
problems  when  evaluating  polices  in  the  public  sector.  Multiple  competing 
stakeholders  often  have  varying  and  conflicting  interests  and  perspectives  on  the 
goals  and  measurement  criteria  of  public  programs.  A  spectrum  of  approaches  to 
discerning  evaluation  criteria  exist.  A  framework  for  policy  analysis  is  established 
based  on  the  basic  components  of  several  models  and  approaches.  This 
framework  provides  a  method  for  choosing  a  preferred  course  of  action  amid 
multiple  complex  alternatives 
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III.  METHODOLOGY 


A.  INTRODUCTION 

This  chapter  presents  the  methodology  used  to  conduct  this  study.  The 
highlighted  portions  of  Figure  6  show  the  steps  and  supporting  task  that  are 
covered  in  this  chapter.  It  describes  the  selected  research  focus,  the  primary 
research  methods,  and  the  steps  in  the  evaluation  criteria  development  process 
outlined  in  Chapter  II.  Furthermore,  this  chapter  furnishes  a  detailed  description 
of  how  the  goal  and  evaluation  criteria  mapping  process  is  used  to  develop  a  single 
set  of  criteria.  An  assessment  of  the  extent  to  which  the  policy  alternatives  meet 
the  criteria  is  provided.  The  chapter  concludes  with  an  outline  of  the  stakeholder 
analysis  process  and  how  the  evaluation  criteria  development  process  is  linked  to 
the  data  collection  and  analysis  stages  of  the  study. 

B.  RESEARCH  FOCUS 

The  objective  of  this  thesis  is  to  determine  which  policy  alternative  for 
reengineering  the  DoD’s  personal  property  program  best  satisfies  stakeholder 
criteria.  The  researcher  chose  to  focus  on  efforts  to  reengineer  the  personal 
property  program  due  to  the  subjective  nature  of  this  effort’s  goals,  the  existence 
of  multiple  competing  stakeholders,  and  the  potential  impact  of  an  implemented 
policy  on  servicemembers.  DoD  efforts  to  achieve  significant  change  in  the 
personal  property  program  have  been  hampered  by  a  lack  of  consensus  among 
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stakeholders  as  to  the  problems  with  the  program  and  possible  solutions  to  those 
problems. 
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Each  stakeholder  has  identified  their  own  set  of  priorities  and  goals  for  any 
alternative  to  the  current  personal  property  program.  DoD  has  attempted  to 
represent  their  own  interests  as  well  as  the  interests  of  the  servicemember  in 
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crafting  policy  alternatives.  Industry  has  exercised  its  political  and  market 
influence  in  a  bid  to  ensure  that  their  goals  and  criteria  are  incorporated  into  the 
policy  that  is  finally  implemented.  This  trichotomy  of  stakes  has  created  a  need  to 
evaluate  the  various  policy  alternatives  fi’om  the  perspectives  of  each  dominant 
stakeholder  group  as  well  as  in  aggregate. 

Relevant  research  and  analysis  provides  a  framework  to  define  a  single  set 
of  criteria  that  can  be  used  to  evaluate  the  current  personal  property  program 
policy  alternatives  from  the  various  stakeholders’  perspectives.  This  effort 
employs  the  basic  tools  of  evaluation  research.  The  design  involves  collecting 
individual  opinions  and  archival  data  using  primary  and  secondary  sources  on 
three  alternate  proposals  to  reengineer  the  personal  property  program.  The  process 
of  developing  evaluation  criteria  used  in  this  thesis  is  broken  down  into  four 
iterative  steps: 

1 .  Identify  goals  of  the  personal  property  program  reengineering  effort 

2.  Establish  evaluation  criteria 

3.  Assess  three  policy  alternatives  developed  by  DoD  for  reengineering 
DoD’s  Personal  Property  Shipment  and  Storage  Program 

4.  Conduct  stakeholder  analysis  and  calculate  the  values  each 
stakeholder  group  places  on  evaluation  critera. 

C.  IDENTIFY  GOALS  OF  PERSONAL  PROPERTY  PROGRAM 

REENGINEERING  EFFORT 

As  a  result  of  the  initial  joint  DoD/industry  working  group  session,  DoD 

and  industry  agreed  to  ten  goals  for  the  reengineered  personal  property  program 

(GAO/NSIAD-97-49,  p.  3).  In  addition  to  these  goals,  government,  industry  and 
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government  contractors  have  also  stated  or  published  goals  for  reengineering  the 
program.'*  Figure  7  summarizes  these  goals.  Although  these  goals  are  easily 
understood,  they  are  too  vague  to  lead  to  agreement  that  they  either  have  or  have 
not  been  met.  Any  assessment  of  the  extent  to  which  each  proposal  meets  the 
goals  is  necessarily  limited  by  the  lack  of  precise  definitions  of  each  goal  and  the 
way  to  achieve  it  (GAO/NSIAD-97-49,  p.  3).  “Moreover,  the  [government  and 
industry  program]  proposals  were  written  in  such  a  way  that  did  not  specifically 
address  how  each  would  achieve  the  stated  goals  (GAO/NSIAD-97-49,  p.  3).” 

In  order  to  discriminate  the  goals  of  the  three  pilots  or  proposed  policy 
alternatives,  it  is  necessary  to  determine  which  organization  or  group  has  the 
ability  to  ensure  its  goals  were  sustained  by  the  alternate  polices.  Government 
policy  development  is  not  only  analytically  complex  but  also  intensely  political, 
involving  many  powerful  interests  and  convictions  that  will  attempt  to  influence 
the  goals  of  any  effort  to  change  policy  (Bonser,  McGregor,  and  Oster,  p.  18). 
DoD  initially  developed  three  pilot  projects  that  emphasized  their  goals  for  a 
future  personal  property  program.  After  numerous  debates,  industry  protests  and 
GAO  reports,  DoD  received  Congressional  approval  to  conduct  its  pilots  projects. 


Primary  sources  include  speeches  by  public  officials,  government  and  industry  congressional  testimony, 
GAO  reports,  and  internal  documents  and  memoranda  of  the  DoD. 
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Figure  7.  Goals  for  Reengineering  Personal  Property  Program 


However,  even  with  congressional  approval,  DoD  still  found  it  necessary  to 
recognize  the  needs  of  industry.  As  a  result,  the  various  DoD  and  industry  goals 
must  be  synthesized  into  concepts  or  general  goals  that  recognize  industry’s 
concerns  as  well  as  fulfill  DoD’s  primary  goals  to  improve  quality  of  life.  Three 
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general  goals  were  identified:  1)  improve  servicemember’s  quality  of  life,  2) 
reduce  the  total  cost  of  the  personal  property  program,  and  3)  maintain  a 
competitive  market  and  limit  the  impact  of  a  policy  change  on  small  businesses. 
These  goals  formed  the  basis  of  the  forward  mapping  process  to  be  detailed  later. 

D.  ESTABLISH  EVALUATION  CRITERIA 

Evaluation  criteria  must  be  able  to  measure  the  extent  to  which  the  goals  or 
concepts  of  a  policy  are  achieved.  “A  good  criterion  provides  a  basis  for 
measuring  progress  toward  achieving  a  goal  (Weimer  and  Vining,  p.  223).” 
However,  not  every  goal  can  be  reasonably  quantified  by  a  single  objective  or 
criterion.  “Rather  than  emphasize  a  single  objective  that  only  measures  one 
dimension  of  a  goal,  you  should  try  to  specify  criteria  that  cover  all  the  important 
dimensions  (Weimer  and  Vining,  p.  223).”  As  Weimer  and  Vining  point  out, 
multiple  criteria  may  be  used  to  measure  the  extent  to  which  a  policy’s  goals  are 
achieved.  “The  set  of  criteria  should  capture  all  the  important  dimensions  of  the 
relevant  goals  (Weimer  and  Vining,  p.  225).” 

The  evaluator  must  be  able  to  systematically  link  the  goals  of  the  policy 
with  the  supporting  sub-goals  and  criteria.  There  are  a  number  of  methods  that 
can  be  used  to  establish  the  relationship  between  goals  and  criteria.  One  such 
method  is  labeled,  “forward  mapping  (Bryson,  1995,  p.  287).”  Through  the  use 
of  forward  mapping  and  similar  techniques  the  evaluator  is  able  to  develop  a  map 
linking  the  program’s  goals  with  its  supporting  sub-goals  and  criteria.  The 
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forward  mapping  process  begins  with  a  program’s  goals,  with  as  clear  a  statement 
as  possible  of  the  policy  maker’s  intent,  and  proceeds  through  a  sequence  of 
increasingly  more  specific  steps  to  define  performance  measures  (Bryson,  p.287). 
At  the  bottom  of  the  process,  one  states,  again  with  as  much  precision  as  possible, 
the  criteria  to  be  used,  measured  in  terms  of  the  ordinal  statement  of  intent 
(Bryson,  p.  287).  The  end  result  of  this  effort  is  the  development  of  a  map  that 
defines  the  policy’s  goals  and  the  characteristics  that  support  achieving  those 
goals. 

Three  general  goals  are  defined  that  embody  the  intent  of  the  reengineering 
effort.  These  goals  are  improve  quality  of  life,  reduce  total  cost,  and  reduce  the 
impact  on  small  business.  Sub-goals  are  identified  that  support  the  attainment  of 
these  general  goals.  Criteria  are  established  based  on  these  sub-goals.  The 
evaluation  criteria  must  be  able  to  measure  the  extent  to  which  each  pilot  project 
manifests  the  goals  and  corresponding  sub-goals.  Therefore,  each  pilot  must  be 
assessed  in  order  to  determine  if  the  criteria  are  adequate  and  are  representative  of 
the  scope  of  the  pilot  projects.  There  is  no  attempt  to  identify  criteria  beyond  the 
scope  of  these  three  alternative  policies.  The  mapping  process  is  useful  in 
reducing  the  complexity  of  the  goals  and  sub-goals  to  the  point  that  the 
performance  measures  or  criteria  are  evident.  Evaluation  criteria  are  established  at 
the  lowest  level  of  indenture.  The  results  of  the  “forward  mapping”  process  are 
summarized  in  Figure  6. 
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The  adequacy  of  the  criteria  is  verified  using  two  common  research 
methods.  The  first  is  the  collection  of  data  through  unobtrusive  measures, 
specifically  archival  research,  and  the  second  is  the  collection  and  verification  of 
data  through  electronic  correspondence. 

Primary  sources  are  used  in  this  study  to  identify  the  goals  and  sub-goals  of 
each  policy  alternative.  Primary  sources  include  speeches,  congressional 
testimony,  reports,  and  documents  issued  by  various  U.S.  government  agencies. 
Examples  include  reports  by  the  General  Accounting  Office,  inter-agency 
information  papers,  briefs  and  memoranda,  and  contractual  statements  of  work. 
Secondary  sources  are  used  to  clarify  specific  aspects  and  characteristics  of  the 
program.  For  example,  the  terms  “best  value”  and  “commercial  business 
practices”  were  used  indiscriminately.  As  a  result,  it  was  necessary  to  define  the 
limits  of  each  as  it  related  to  the  pilot  projects. 

Data  collected  fi'om  archival  sources  served  as  a  basis  for  the  questions 
asked  through  electronic  correspondence.  Data  were  requested  via  electronic  mail 
to  verify  the  accuracy  of  the  information  gained  firom  primary  or  secondary 
sources.  Data  were  collected  firom  a  limited  group  of  individuals  having  expert 
knowledge  of  the  subject  area  due  to  their  position  or  experience.  Correspondence 
also  provided  additional  information  that  was  not  gathered  from  archival 
resources.  This  information  provided  insight  into  known  goals  and  sub-goals  and 
highlighted  unknown  features  of  each  pilot. 
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I.  Improve  Quality  of  Life.  This  goal  is  based  on  achieving  customer  satisfaction  and 
improving  or  expanding  services  to  the  customer. 

A.  Use  of  Federal  Acquisition  Regulation  rFARVbased  contracting  method 

B.  Loss  and  Damage  Claim 

1 .  Full  Replacement 

2.  Direct  Settlement  with: 

a.  3"^^  Party 

b.  Transportation  provider 

c.  Government 

3.  Reduced  claim  preparation  requirements 

4.  Reduced  claim  settlement  period 

C.  Provider  Qualification  -  Standards  for  contract  award 

1 .  Financial  stability  -  providers  have  strong  financial  credentials,  low  risk  of 
contractor  failure 

2.  Ensured  capacity  -  proven  track  records  and  capability  proportionate  to  the 
shipper’s  requirements. 

3.  Past  Performance  -  imposition  of  stringent  performance  standards 

•  On-time  pick-up  and  delivery  rates 

•  Loss  and  damage  rates  comparable  to  or  better  than  the  lowest  commercial 
account’s  loss  and  damage  rates 

•  Customer  feedback  (intangibles)  -  i.e.,  professional  crews,  comfort  with 
provider 

4.  Inclusion  of  Service  Contract  Act  -  this  Act  requires  the  Providers  to  pay 
employees  the  prevailing  minimum  wage  for  the  area  in  which  they  are  working 

5.  Inconvenience  payments  -  provider  pays  servicemember  in  the  event  that  the 
provider  is  unable  to  deliver  personal  property  when  scheduled 

D.  Provider  Performance  Measurement  -  Relationship  with  providers  contingent  on 
them  meeting  and  maintaining  performance  standards 

Use  of  customer  surveys 

Random  inspections 

Periodic  performance  assessment 

E.  Move  Management 

1.  Pre-move  counseling 

2.  Single  point  of  contact  move  management 

3.  Binding  estimates  for  excess  costs 

4.  Destination  Information 

5.  Shipment  tracking 

6.  Delivery  coordination 

7.  Real  Estate  assistance 

Figure  8.  Goals  and  Evaluation  Criteria 
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II.  Reduce  Total  Cost.  This  concept  includes  direct  (transportation,  storage,  accessorial) 
and  indirect  cost.  The  indirect  baseline  cost  includes  management  of  the  personal  property 
program,  personal  property  services  provided,  quality  assurance,  financial  transaction 
processing,  and  additional  systems  operations  and  maintenance  cost.  This  concept  is  based  on 
reducing  the  government’s  administrative  burden  through  eliminating  or  improving  current 
processes  or  implementing  better  business  practices. 

A.  Pricing  Strategy  -Improve  rate  solicitation  process  and  adopt  commercial  or  standard 
rate  structure 

1 .  Continued  use  of  DoD  rate  solicitations,  domestic  and  international 

2.  Adoption  of  commercial  tariff,  domestic 

3 .  GSA  Tender  1-W,  domestic  and  international 

4.  Single-Factor-Rate  (SFR),  domestic  and  international 

5.  Commissions-  paid  to  move  managers  by  transportation  providers 

B.  Traffic  Allocation  -  Allocation  of  traffic  to  and  amongst  providers 

1 .  Contract  consolidation  -  consolidation  of  2  or  more  procurement  requirements 
for  goods  or  services  previously  provided  or  performed  under  separate  small 
contracts  into  a  solicitation  of  offers  for  a  single  contract 

2.  Traffic  distribution  based  on  quality 

3.  Traffic  distribution  performed  by: 

a.  3"^^  Party 

b.  Servicemember 

c.  Government 

4.  Reduction  or  elimination  of  current  use  of  the  Traffic  Distribution  Record 
(TDR) 

5.  Fewer  contracts  awarded 

6.  Longer  contract  periods 

C.  F.DT  Strategy:  EDI  capability  or  a  willingness  to  do  business  in  an  EDI  environment 

D.  Reduction  in  the  complexity  of  the  Movement  Category  and  Service  Structure 

III.  Reduce  Impact  on  Small  Business.  This  concept  will  be  measured  against  the  Small 
Business  Act,  Public  Law  85-536,  as  amended,  which  states  that  the  government-wide  goal 
for  participation  by  small  business  concerns  shall  be  established  at  not  less  than  23  percent 
revenue.  Small  business  is  defined  for  purposes  of  this  evaluation  plan  as  those  businesses 
with  annual  average  gross  revenue  of  $18.5  million  or  less  (prime  contractor  or  Provider). 
This  concept  strives  to  develop  and  maintain  a  competitive  market. 

A.  Traffic  Structure:  combines  traffic  flow  and  traffic  scope-focus  on  streamlining 
existing  network  based  on  outboimd  traffic  lanes  from  an  area  of  responsibility 
(AOR)  to  a  rate  area  (RA) 

B.  Payment 

1 .  Defense  Finance  &  Accounting  Service  (DFAS) 

2.  Bank  Service  Contract 

3.  Government  Credit  Card:  EMPAC 

C.  SBA  Compliance  Means 

1 .  Application  of  Small  Business  Act 

2.  Small  Business  Set  Asides  as  Primes 

Figure  8.  Goals  and  Evaluation  Criteria  (cont.) 
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The  use  of  archival  sources  and  electronic  correspondence  has  certain 
limitations.  First,  archives  may  contain  errors,  be  incomplete  or  be  no  longer 
relevant.  Secondly,  the  source  may  present  only  one  perspective.  Finally, 
different  meanings  may  be  assigned  to  similar  terms  or  concepts  based  on  the 
archival  source  or  respondent.  These  problems  were  mitigated  by  using  up  to  date 
multiple  sources.  Multiple  sources  from  different  levels  and  components  of  DoD 
and  government  were  reviewed.  Furthermore,  archival  sources  and  electronic 
responses  were  cross-referenced.  For  example,  congressional  testimonies  were 
checked  for  consistency  with  GAO  reports.  Also,  electronic  responses  were  used 
to  verify  accuracy  and  relevancy  of  internal  memoranda,  statements  of  work,  and 
performance  work  statements. 

E.  ASSESS  POLICY  ALTERNATIVES 

This  step  determines  the  extent  to  which  each  policy  alternative  fulfills  the 
established  criteria.  The  results  of  the  forward  mapping  process  summarized  in 
Figure  8  were  used  to  develop  criteria  by  which  to  assess  the  policy  alternatives. 
There  are  two  steps  in  this  process.  The  first  step  is  the  identification  of  general 
goals  and  the  specification  of  sub-goals.  In  the  second  step,  criteria  were 
developed  to  measure  the  extent  to  which  the  goals  and  sub-goals  are  achieved. 
Additionally,  the  policy  alternatives  were  compared  and  appraised  at  the  criteria 
and  sub-criteria  level  where  data  were  available. 
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Thus,  the  forward  mapping  process  creates  a  strong  relationship  between 
the  goals,  supporting  sub-goals  and  the  evaluation  criteria.  For  example,  in  Figure 
9,  there  are  five  ways  to  improve  quality  of  life  for  the  servicemember:  A) 
contracting  method,  B)  loss  and  damage  claims,  C)  provider  qualifications,  D) 
provider  performance  measurement,  and  E)  move  management.  There  are  three 
criteria  for  measuring  the  extent  to  which  provider  performance  measures  have 
been  established:  1)  customer  surveys,  2)  random  inspections,  and  3)  periodic 
performance  measurement. 

Once  the  goals  and  criteria  were  identified  and  their  relationship  to  each 
other  was  established,  I  created  a  scale  to  measure  the  extent  to  which  each  policy 
alternative  embodied  the  evaluation  criteria.  The  following  scale  was  used  to  rate 
how  important  the  criteria  were  to  the  pilot  project: 

1  =  Not  important 

2  =  Little  importance 

3  =  Medium  importance 

4  =  High  importance 

5  =  Critical 

Using  this  scale,  I  independently  rated  each  pilot  project  with  respect  to  the 
criteria.  I  made  a  judgment  as  to  the  value  each  policy  alternative  places  on  a 
criterion.  The  pilot  project  characteristics  displayed  in  Figure  2  formed  the  basis 
of  this  assessment.  Statements  of  work  and  performance  work  statements  were 
also  used  to  provide  details  on  specific  aspects  of  each  pilot. 
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Goal 


Sub-goal _ 

A.  Contracting  Method 


Criteria _ 

FAR  -based _ 

1 .  Full  Replacement 


Sub-criteria 


L  Improve 
Quality  of  Life 

B.  Loss  and  Damage  Claim 

2.  Direct  Claim  Settlement: 

a.  Move  Manager 

b.  Transportation 
Provider 

c.  Government 

3.  Reduced  claim  preparation 
requirements 

4.  Reduced  claim  settlement  period 

C.  Provider  Qualification 

1.  Financial  stability 

2.  Ensured  capacity 

3.  Past  performance 

4.  Inclusion  of  Service  Contract  Act 

5.  Inconvenience  payments 

D.  Provider  Performance 
Measurement 

3.  Periodic  Performance  Assessment 

E.  Move  Management 

1 .  Pre-move  counseling 

2.  Single  point  of  contact  move 
management 

3.  Binding  estimates  for  excess  costs 

4.  Destination  information 

6.  Delivery  coordination 

7.  Real  Estate  assistance 

IL  Reduce 

Total  Cost 

A.  Pricing  Strategy 

1.  Continued  use  of  DoD  solicitations 

Domestic 

International 

2.  Commercial  Tariff 

Domestic 

3.  GSA  Tender  1-W 

Domestic 

International 

4.  Single-Factor-Rate 

Domestic 

International 

5.  Commissions  for  move  manager 

B.  Traffic  Allocation 

1 .  Contract  consolidation 

2.  Traffic  distribution  based  on  quality 

3.  Traffic  distribution  performed  by: 

a.  Move  Manager 

b.  Servicemember 

c.  Government 

4.  Reduction  or  Elimination  of  Traffic 
Distribution  Record 

5.  Reduction  in  Contracts  awarded  - 

6.  Increase  length  of  Contract  award 

C.  EDI  capability  required 

iiiimiiiiiiiiiiiiiii^ 

D.  Service  Structure 

Reduction  in  movement  and  storage 
categories 

III.  Limit 

Impact  on 

Small 

Business; 
Maintain  a 
Competitive 
Market 

A.  Traffic  Structure 

Change  current  structure  of  AOR  to 

RA 

B.  Payment 

l.DFAS 

2.  Bank  Service  Contract 

3.  Government  Credit  Card 

C.  SBA  Compliance  Means 

1 .  Application  of  Small  Business  Act 

2.  Small  Business  prime  contractor  set 
asides 

Figure  9.  Goals  and  Evaluation  Criteria 
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Figure  10  lists  the  values  assigned  for  each  criterion  relative  to  the  pilot 
projects.  These  values  represent  how  important  each  criterion  is  to  a  particular 
policy  alternative.  For  example,  the  SAM  pilot  places  medium  importance  on  the 
use  of  FAR-based  contracts.  On  the  other  hand,  FAR-based  contracts  are  highly 
important  in  the  FSMP  and  critical  to  the  MTMC  pilot. 

Interaction  through  electronic  correspondence  with  the  project  officers  for 
each  pilot  project  biased  this  assessment  somewhat.  Certain  project  officers  were 
better  able  to  articulate  the  characteristics  and  benefits  of  their  pilot.  In  an  attempt 
to  reduce  this  bias,  I  limited  my  interaction  with  the  project  officers.  This  step  is 
performed  prior  to  determining  stakeholder  values  in  order  to  prevent  stakeholder 
comments  from  further  biasing  this  study. 

The  lack  of  inter-rater  reliability  also  creates  a  bias  in  this  assessment. 

Only  one  rater  was  available  to  conduct  the  assessment  of  the  policy  alternatives. 
This  limitation  is  a  result  of  the  complex  nature  of  the  personal  property  program 
and  the  reengineering  effort.  In  order  to  rate  the  pilots,  a  level  of  knowledge  and 
familiarity  with  the  program  is  required.  The  use  of  multiple  raters  in  follow-on 
research  will  provide  a  greater  degree  of  accuracy  and  reliability  to  this 
assessment. 
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Sub-goals 

Evaluation  Criteria 

Pilots 

SAM 

FSMP 

MTMC 

Contracting  Method 

FAR  -based 

3 

4 

5 

Loss  and  Damage 
Claim 

Full  Replacement 

5 

5 

4 

Direct  Claim  Settlement 

Move  Manager 

1 

5 

1 

Transportation  Provider 

5 

1 

4 

Government 

2 

1 

3 

Reduced  claim  preparation  requirements 

2 

4 

4 

Reduced  claim  settlement  period 

4 

5 

3 

Provider 

Qualification 

Financial  stability 

2 

4 

3 

Ensured  capacity 

1 

4 

4 

Past  performance 

4 

5 

3 

Inclusion  of  Service  Contract  Act 

4 

1 

1 

Inconvenience  payments 

4 

4 

4 

Provider 

Performance 

Measurement 

Customer  Surveys 

5 

4 

4 

Random  Inspections 

3 

3 

3 

Periodic  Performance  Assessment 

1 

3 

5 

Move  Management 

Pre-move  counseling 

3 

5 

3 

Single  point  of  contact  move  management 

1 

5 

1 

Binding  estimates  for  excess  costs 

1 

4 

2 

Destination  information 

1 

4 

1 

Shipment  tracking 

3 

5 

4 

Delivery  coordination 

5 

3 

2 

Real  Estate  assistance 

1 

4 

1 

Pricing  Strategy 

Continued  use  of  DoD 
solicitations 

Domestic 

1 

1 

1 

International 

1 

1 

1 

Commercial  Tariff 

Domestic 

5 

5 

1 

GSA  Tender  1-W 

Domestic 

1 

1 

1 

International 

1 

1 

1 

Single-Factor-Rate 

Domestic 

1 

1 

1 

International 

1 

5 

1 

Commissions  for  move  manager 

1 

3 

1 

Traffic  Allocation 

Contract  consolidation 

1 

5 

4 

Traffic  distribution  based  on  quality 

4 

4 

3 

Traffic  distribution 
performed  by: 

Move  Manager 

1 

5 

,  1 

Servicemember 

4 

1 

1 

Government 

2 

1 

5 

Reduction  or  Elimination  of  Traffic  Distribution  Record 

3 

5 

3 

Reduction  in  Contracts  awarded 

1 

5 

3 

Increase  length  of  Contract  award 

1 

4 

4 

EDI  capability  required  | 

1 

3 

5 

Service  Structure 

Reduction  in  movement  and  storage  categories 

3 

1 

I 

Traffic  Structure 

Change  current  structure  of  AOR  to  RA 

1 

3 

4 

Payment 

DFAS 

1 

1 

5 

Bank  Service  Contract 

1 

5 

1 

Government  Credit  Card 

5 

1 

1 

SBA  Compliance 
Means 

Application  of  Small  Business  Act 

4 

4 

4 

Small  Business  prime  contractor  set  asides 

5 

1 

1 

Figure  10.  Value  of  Criteria  for  Each  Pilot  Project 
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F.  STAKEHOLDER  ANALYSIS  PROCESS 

Once  the  evaluation  criteria  are  determined,  the  evaluator  must  establish  the 
value  to  be  placed  on  each  criterion.  Attempts  to  introduce  policy  analysis  into 
most  arenas  of  public  policy  inevitably  founder  on  evident  disagreements  over 
both  the  measurement  techniques  and  value  criteria  (Fischer  and  Forester,  p.232). 
In  undertaking  their  work,  evaluators  usually  find  themselves  confronted  with 
individuals  and  groups  who  hold  competing  and  sometimes  combative  views  on 
the  policy  alternatives  and  whose  interests  will  be  affected  by  the  outcome  (Rossi 
and  Freeman,  p.422).  These  individuals  and  groups  often  place  vastly  different 
values  on  evaluation  criteria.  Smith  points  out  that  “it  is  rare  to  find  consensus 
about  what  constitutes  the  objectives  of  a  public  sector  program  (Smith,  1996, 

P-9)  ” 

In  order  to  establish  whether  consensus  exists  or,  when  no  consensus  exists, 
the  relative  weight  or  value  to  assign  each  criterion,  the  evaluator  must  perform  a 
stakeholder  analysis.  The  identification  of  key  stakeholders  and  the  determination 
of  a  set  of  stakeholder  criteria  are  critical  to  choosing  a  preferred  alternative  from 
amongst  complex  policy  options.  “If  an  organization  does  not  know  who  its 
stakeholders  are,  what  criteria  they  use  to  judge  the  organization,  and  how  the 
organization  is  performing  against  those  criteria,  there  is  little  likelihood  that  the 
organization  will  know  what  is  should  do  to  satisfy  its  key  stakeholders  (Bryson, 
p.  70).”  The  stakeholder  analysis  process  is  broken  down  into  two  stages  for  the 
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purposes  of  this  study:  1)  identifying  the  stakeholders  and  2)  determining  the  value 
they  place  on  each  objective. 

1.  Identification  of  Stakeholders 

The  first  step  in  a  stakeholder  analysis  is  to  identify  an  organization’s 
stakeholders.  This  step  usually  begins  with  the  construction  of  a  stakeholder  map 
(Roberts  and  King,  1989,  p.  65).  Mitroff  defines  stakeholders  as  “all  those  parties 
who  either  affect  or  are  affected  by  an  organization’s  actions,  behavior,  and 
policies  (Mitroff,  1983,  p.  4).”  This  definition  is  very  similar  to  the  definition 
used  by  Freeman.  Freeman  defines  stakeholders  as  “any  group  or  individual  who 
can  affect  or  is  affected  by  the  achievement  of  an  organization’s  objectives 
(Freeman,  1984,  p.  39).”  Both  definitions  convey  essentially  the  same  meaning 
and  are  used  interchangeably  in  this  study. 

The  purpose  of  the  stakeholder  map  is  to  identify  the  stakeholders  in  the 
reengineering  of  the  personal  property  program.  An  initial  stakeholder  map  must 
not  leave  out  any  group  or  individual  that  can  affect  or  is  affected  by  an 
organization’s  policies  (Freeman,  p.  52).  Roberts  and  King  recommend  reducing 
the  number  of  stakeholders  by  categorizing  them  into  larger,  more  general  groups 
(Roberts  and  King,  p.  69).  Once  identified,  individual  stakeholders  are  grouped 
according  to  similar  stakes  and  organized  into  an  overall  “stakeholder  map” 
(Roberts  and  King,  p.69). 
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Three  stakeholder  groups  relevant  to  reengineering  the  personal  property 
program  were  identified.  Figure  1 1  contains  a  list  of  the  dominant  stakeholder 
groups.  These  groups  are  1)  government,  2)  industry,  and  3)  servicemembers. 
Industry  is  broken  down  further  into  two  sub-groups:  relocation  management  and 
transportation  providers. 


Government 

Industry 

.  OSD 

1.  Transportation 

•  USTRANSCOM 

Providers 

•  MTMC 

2.  Relocation 

•  Army 

Personal  Property 

Managers 

•  Navy 

f  Program  ^ 

V  Reengineering  Effort  J 

Setvicemember 


Figure  11.  Stakeholder  Map 
a.  Government  Stakeholders 

The  United  States  Government  is  responsible  for  initiating  and 
funding  the  program  as  well  as  for  deciding  which  policy  alternative  is  instituted. 
There  are  numerous  stakeholders  within  the  government.  The  primary  stakeholder 
is  the  Department  of  Defense.  Agencies  and  components  of  DoD  are  also 
classified  as  stakeholders  to  the  reengineering  effort. 
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b.  Industry  Stakeholder 

Industry  stakeholders  are  commercial  organizations  and  groups  who 
compete  for  business  under  the  personal  property  program.  There  are  two  major 
sub-groups  within  the  industry  stakeholder  group.  These  are  1)  relocation 
management  companies  and  2)  transportation  providers. 

(1)  Relocation  Managers.  Relocation  managers  provide  a 
wide  range  of  relocation  services  through  an  integrated  network  of  transportation 
providers,  real  estate  agents,  and  move  coordinators.  The  Military  Mobility 
Coalition  is  an  ad  hoc  organization  formed  in  the  spring  of  1996  to  represent  the 
relocation  management  industry’s  interests  in  working  with  the  DoD’s  household 
goods  reengineering  initiatives  (Groover,  Jr.,  1999).  The  majority  of  membership 
consists  of  relocation  and  move  management  companies.  The  coalition  currently 
includes  twenty-seven  military  organizations.  Business  Executives  for  National 
Security,  and  most  major  real  estate  networks  (Groover,  Jr.,  1999). 

(2)  Transportation  Providers.  Two  organizations  represent 
and  promote  the  interests  of  the  transportation  and  storage  portions  of  the  moving 
industry  before  federal  and  state  legislative  and  regulatory  bodies.  These  are  the 
Household  Goods  Forwarders  Association  (HGFA),  and  the  American  Moving 
and  Storage  Association  (AMSA).^  The  members  of  the  HGFA  comprise  all 
facets  of  the  moving  industry,  including  moving  and  storage  firms,  national  van 

^  The  American  Moving  and  Storage  Association  was  formed  when  the  American  Movers  Conference  and 
the  National  Moving  and  Storage  Association  combined. 
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lines,  line-haul  firms,  international  and  domestic  forwarders,  port  agents,  claims 
adjusters,  material  providers,  insurance  providers,  technology  firms,  and  others 
(Head,  1999).  Members  include  some  1,500  international  and  domestic 
companies,  the  majority  of  which  are  small  businesses  (Head,  1999).  The  AMSA 
is  a  national  trade  association  of  the  moving  and  storage  industry.  It  has 
approximately  3,400  members  nationwide  representing  the  entire  spectrum  of  the 
industry  and  including  national  van  lines,  their  affiliated  agents  (carriers  and  non¬ 
carriers)  and  independent  regional  and  national  carriers  (Harrison,  1999). 
c.  Servicemember 

Servicemembers  are  the  customers  of  the  personal  property  program. 
There  are  over  1.4  million  servicemembers  in  the  United  States  Armed  Forces 
serving  in  140  countries  worldwide.  There  are  an  additional  1.9  million  family 
members.  The  majority  of  servicemembers  are  under  the  age  of  twenty-five  years 
old  and  have  served  less  than  10  years. 

2.  Determining  Values  Stakeholders  Place  on  Evaluation  Criteria 
The  second  step  of  the  stakeholder  analysis  is  to  determine  how  important 
an  objective  is  to  each  stakeholder  and  respective  stakeholder  group.  “Different 
stakeholders  do  not  generally  share  the  same  definitions  of  an  organization’s 
‘problems,’  and  hence,  they  do  not  in  general  share  the  same  ‘solutions  (Mitroff, 
p.  5).’“  Therefore,  an  evaluator  must  strive  to  determine  the  value  stakeholders 


place  on  the  objectives,  and  the  associated  measurement  criteria,  that  formulate  a 
given  “solution”  or  policy. 

Conflict  over  values  arises  because  the  maximization  of  one  value 
inevitably  is  at  the  expense  of  another  (Fischer  and  Forester,  p.  232).  Evaluators 
are  often  unsure  whose  perspective  they  should  take  in  designing  an  evaluation 
(Rossi  and  Freeman,  p.  425).  As  noted  in  Chapter  II,  there  is  no  proper 
perspective,  but  rather  different  perspectives  may  be  equally  legitimate.  The 
responsibility  of  the  evaluator  is  not  to  take  one  of  the  many  perspectives  as  the 
legitimate  one,  but  rather  to  be  clear  from  which  perspectives  a  particular 
evaluation  is  being  undertaken  (Rossi  and  Freeman,  p.  425). 

In  many  cases  the  information  required  to  complete  this  step  is  readily 
available  from  archival  records.  “In  some  cases,  however,  the  information  must 
be  systematically  gathered  by  interviewing  individual  stakeholders  (Freeman,  p. 

1 14).”  Archival  sources  provided  background  on  the  perspectives  of  the  various 
stakeholder  groups.  However,  interviews  were  necessary  to  determine  the  value 
stakeholders  place  on  the  evaluation  criteria. 

a.  Developing  a  Questionnaire 

Once  the  initial  stakeholder  analysis  was  completed,  I  developed  a 
questionnaire  to  ascertain  the  value  that  stakeholders  place  on  the  evaluation 
criteria.  There  is  no  attempt  to  identify  stakeholder  objectives  that  fall  outside  the 
scope  of  the  three  alternative  policies.  A  copy  of  the  questionnaire  is  provided  in 
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Appendix  B.  Questions  were  phrased  in  such  a  way  as  to  necessitate  a  structured 
response  from  each  interviewee.  The  questionnaire  provides  a  single  format  by 
which  to  evaluate  responses  from  the  sample. 

b.  Choosing  a  Sample 

In  order  to  conserve  time  and  money,  a  convenience  sample  was 
used.  Interviewees  were  chosen  from  each  stakeholder  group.  A  total  of 
seventeen  interviews  were  conducted  with  representatives  from  the  three 
stakeholder  groups.  Seven  interviews  were  conducted  with  representatives  from 
the  government,  four  interviews  were  conducted  with  members  of  industry,  and 
six  interviews  were  conducted  with  servicemembers  or  servicemember 
representatives. 

Interviews  were  conducted  with  government  officials  at  every  level 
of  the  reengineering  effort  beginning  with  USTRANSCOM  and  ending  at  the 
project  management  level.  Interviews  with  members  of  industry  were  divided 
equally  between  transportation  providers  and  relocation  managers.  All 
servicemembers  interviewed  were  officers  in  the  United  States  Marine  Corps  with 
six  to  fifteen  years  of  active  service  and  having  moved  an  average  of  seven  times. 
A  representative  of  a  servicemember  organization  was  also  interviewed. 
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c.  Conducting  the  Interviews 

Interviews  were  conducted  telephonically.  Each  interviewee  was 
told  the  goal  of  the  interview.  The  interviewee  was  then  asked  how  important  they 
felt  each  criterion  was  to  any  effort  to  change  the  current  personal  property 
program.  The  respondents  answered  the  questions  using  the  following  scale: 

0  =  Not  applicable 

1  =  Not  important 

2  =  Low  importance 

3  =  Medium  importance 

4  =  High  importance 

5  =  Critical 

Definitions  for  the  evaluation  criteria  were  based  on  the  goals  and 
criteria  described  in  Figure  8.  During  the  course  of  the  interviews,  respondents 
often  provided  additional  comments  and  information  beyond  the  scope  of  the 
question.  This  information  was  useful  in  developing  a  more  thorough 
understanding  of  the  issues  involved  in  the  reengineering  effort  and  provided 
insight  into  the  political  dynamics  of  public  policy  evaluation  and  implementation. 
These  comments  are  discussed  in  Chapter  VI,  Conclusions  and  Recommendations. 

d.  Classifying  Results 

Collected  data  were  coded  into  a  spreadsheet  and  measures  of 
central  tendency  were  calculated  for  each  evaluation  criterion,  including  arithmetic 
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means  and  standard  deviations.  Using  the  measures  generated  from  the  data, 
additional  measures  were  compiled  at  the  sub-goal  and  goal  level  for  review  and 
examination.  This  data  is  presented  in  Chapter  IV.  An  analysis  of  the  results  of 
the  statistical  results  is  provided  in  Chapter  V. 

e.  Validation 

If  the  evaluation  process  is  to  be  enhanced,  the  data  gathered  must  be 
valid;  that  is,  the  “right”  responses  must  be  assessed,  and  in  a  manner  that  will 
elicit  meaningful  measurements  (Levine,  Berenson,  and  Stephan,  1998,  p.  9). 
Interview  data  must  be  validated  as  far  as  possible  to  ensure  the  accuracy  of  the 
results.  Misunderstandings  may  be  introduced  because  of  difficulties  in 
establishing  proper  modes  for  communicating  with  different  stakeholders  (Rossi 
and  Freeman,  p.  425).  Stakeholders  define  terms  in  different  fashions  and  their 
responses  are  based  on  that  definition.  Questions  were  to  provide  each 
interviewee  with  same  definition  of  each  criterion.  However,  interviewees  had 
different  degrees  of  involvement  and  familiarity  with  the  alternative  policies  and 
characteristics  of  the  personal  property  program.  In  summary,  although  the 
methodology  was  to  apply  standard  definitions  to  all  the  criterion,  interviewees 
often  required  additional  clarification  beyond  the  standard  definitions. 

Another  potential  problem  is  the  reliability  of  the  results.  Bryson 
points  out  that  “stakeholders  may  not  be  completely  honest  (Bryson,  p.  73).” 
Responses  are  the  interviewees’  professed  or  espoused  values,  not  necessarily  the 
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true  value  they  place  on  the  criteria.  Value  is  also  relative  amongst  respondents. 
Although  the  scale  was  defined,  my  speculation  is  that  some  respondents  may 
have  inflated  their  values  while  others  deflated  theirs.  This  conclusion  is  based  on 
the  general  comments  some  interviewees  made  with  respect  to  the  criteria.  In 
these  cases,  I  reminded  the  interviewee  of  the  scale  being  used.  Sometimes 
interviewees  attempted  to  answer  according  to  the  program  they  supported  and 
sometimes  they  espoused  their  personal  opinions. 

An  attempt  was  made  to  mitigate  potential  sources  of  error  by 
verifying  data  with  archival  sources.  Responses  were  double-checked  to  ensure 
consistency  with  congressional  testimony,  published  stakeholder  documents  and 
GAO  reports.  I  also  reviewed  responses  to  identify  any  obvious  disparate  trends 
amongst  stakeholders  within  a  stakeholder  group. 

G.  CHAPTER  SUMMARY 

The  four-step  method  to  develop  evaluation  criteria  and  establish  the  value 
stakeholders  place  on  each  criterion  is  an  iterative  process.  Evaluation  criteria  are 
developed  using  data  gathered  through  archival  and  electronic  correspondence 
research.  While  many  steps  in  the  process  are  largely  subjective,  a  concerted 
effort  was  made  to  crosscheck  and  verify  each  assumption.  The  result  is  a  list  of 
evaluation  criteria  and  an  assessment  of  the  extent  to  which  each  pilot  project 
meets  those  criteria.  The  value  stakeholders  place  on  each  criterion  is  established 
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by  conducting  interviews  with  a  convenience  sample  of  stakeholders.  This  data  is 
presented  in  Chapter  IV. 
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IV.  DATA 


A.  INTRODUCTION 

This  chapter  presents  and  discusses  the  data  collected  on  the  values 
stakeholders  place  on  the  evaluation  criteria  for  reengineering  DoD’s  personal 
property  program.  The  data  presented  were  gathered  through  telephonic 
interviews  using  a  standard  questionnaire.  The  respondents  were  from  a 
convenience  sample  of  stakeholders.  A  total  of  seventeen  interviews  were 
conducted.  Every  stakeholder  contacted  agreed  to  be  interviewed. 

B.  STATISTICAL  MEASURES 

Interviewee  responses  were  recorded  for  each  evaluation  criteria.  The  data 
were  then  entered  into  a  spreadsheet.  The  arithmetic  mean  and  the  standard 
deviation  of  individual  stakeholder  responses  were  computed  for  each  stakeholder 
group  as  well  as  an  aggregate  for  all  stakeholders.  Summarized  data  reveal 
differences  in  values  between  stakeholders,  as  well  as  expose  shared  values 
amongst  stakeholders.  Interview  data  and  statistical  analysis  is  presented  in 
Appendix  C. 

Data  are  presented  and  discussed  at  the  goal  and  sub-goal  level  as  shown  in 

Figure  9.  I  integrated  evaluation  criteria  data  at  the  sub-goal  level  by  calculating 

the  mean  of  the  group  averages  for  the  criteria  that  support  each  sub-goal.  For 

example,  the  value  assigned  to  “D)  performance  measurement”  by  each 

stakeholder  group  represents  the  average  of  the  values  calculated  for:  1)  customer 
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surveys,  2)  random  inspections,  and  3)  periodic  performance  assessment.  I 
integrated  sub-goal  values  at  the  general  goal  level  by  computing  the  average  for 
the  sub-goal  values  that  support  each  goal.  For  instance,  I  derived  the  value  for  “I) 
improve  quality  of  life”  by  determining  the  average  for  the  following  sub-goals: 

A)  contractor  method,  B)  loss  and  damage  claim,  C)  provider  qualification,  D) 
performance  measurement  and  E)  move  management. 

Statistical  methods  help  reduce  the  “raw”  data  generated  from  the  interview 
process  by  generating  summary  measures.  The  measures  help  to  focus  the  data 
and  reduce  the  variation  between  individual  stakeholders  within  a  stakeholder 
group.  The  use  of  common  statistical  measures  simplifies  the  evaluation  process 
when  numerous  stakeholders  value  multiple  complex  evaluation  criteria 
differently.  However,  an  arithmetic  mean  may  be  skewed  when  extremely  high  or 
low  responses  knovm  as  outliers  are  present  in  the  data. 

1.  Improve  the  Servicemember’s  Quality  of  Life 


Improve  Quality  of  Life 

;i;Stak^|b©ld|r'::i;K|^^^ 

Average 

Std.  Dev. 

Aggregate 

3.68 

0.40 

Government 

Providers 

WBSSM 

1.15 

Relocation 

4.09 

0.57 

Industry 

Servicemembers 

3.65 

HEE&H 

Of  the  three  goals,  stakeholders  place  the  highest  value  on  improving 
servicemembers  quality  of  life.  Clearly,  this  goal  is  seen  as  the  key  component  in 
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the  effort  to  reengineer  the  personal  property  program.  Relocation  providers  place 
the  highest  value  on  this  goal.  Transportation  providers  place  the  lowest  value  on 
this  goal.  Government  and  servicemember  stakeholders  are  the  most  consistent  in 
placing  a  high  value  on  this  goal. 

a.  Contracting  Method 


Contracting  Method 

Stakeholder 

Average 

Std.  Dev. 

Aggregate 

3.03 

0.29 

Government 

3.33 

1.63 

Providers 

1.00 

0.00 

Relocation 

4.50 

0.71 

Industry 

2.75 

2.06 

Servicemembers 

3.00 

2.00 

Transportation  providers  are  diametrically  opposed  to  the  use  of 
FAR-based  contracts  in  the  personal  property  program.  Government  and 
relocation  management  stakeholders  consistently  support  the  use  of  FAR  based 
contracts  with  the  exception  of  one  individual  stakeholder.  Servicemembers,  for 
the  most  part,  are  not  concerned  with  the  type  of  contracting  method. 
b.  Loss  and  Damage  Claim 


Improving  Loss  and  Damage  Claim  Process 

Stakeholder 

Average’ 

std.  Dev. 

Aggregate 

4.11 

0.82 

Government 

4.31 

0.64 

Providers 

3.46 

Relocation 

1.17 

Industry 

3.94 

Servicemembers 

0.91 
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There  is  generally  strong  support  amongst  stakeholder  groups  for 
improving  the  loss  and  damage  claim  process.  Providing  full  replacement 
coverage  for  the  servicemember  during  the  move  received  the  highest  aggregate 
value  and  can  be  considered  critical  to  any  effort  to  reengineer  the  personal 
property  program.  The  standard  deviation  indicates  a  large  variation  in  responses. 
This  variation  is  due  to  the  choices  respondents  had  when  considering  direct  claim 
settlement.  Interviewees  were  asked  to  choose  between  three  mutually  exclusive 
options  when  settling  a  loss  or  damage  claim.  The  majority  of  stakeholders  prefer 
claims  settlement  with  the  transportation  provider.  A  reduced  claim  preparation 
requirement  and  claim  settlement  period  are  both  highly  valued  by  all 
stakeholders. 


c.  Provider  Qualification 


Provider  Qualifications 

IStakelMder,  „ 

vAverage 

Std.r)ev. 

Aggregate 

3.69 

0.66 

Government 

3.90 

0.36 

Providers 

3.40 

1.19 

Relocation 

3.30 

1.40 

Industry 

3.35 

1.21 

Servicemembers 

3.82 

0.63 

Creating  or  increasing  standards  for  awarding  contracts  is  valued  as 
having  medium  to  high  importance  to  the  stakeholders.  Stakeholders  also  perceive 
ensuring  industry  has  the  capacity  to  meet  contractual  obligations  and  qualifying 
providers  based  on  past  performance  as  important  features  to  the  reengineering 
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effort.  The  inclusion  of  the  Service  Contract  Act  is  seen  a  having  very  little  value 
by  industry. 

d.  Performance  Measures 


Use  of  Performance  Measures 

Stakeholder 

Average 

Std*Dev. 

Aggregate 

3.73 

0.34 

Government 

3.71 

0.49 

Providers 

3.83 

0.29 

Relocation 

3.67 

1.04 

Industry 

3.75 

0.50 

Servicemembers 

3.72 

0.10 

Stakeholders  are  fairly  uniform  in  their  feeling  that  there  should  be 
some  form  of  performance  measurement  within  the  personal  property  program. 
Stakeholders  value  the  use  of  customer  surveys  the  most.  However,  periodic 
performance  assessments  are  almost  as  highly  valued. 
e.  Move  Management 


Expanding  Move  Management  Services 

Stakeholder 

Average 

Std.pev. 

Aggregate 

Government 

Providers 

3.43 

Relocation 

4.57 

Industry 

4.00 

Servicemembers 

3.64 

This  sub-goal  includes  the  provision  of  expanded  services  to  the 
servicemember  by  a  move  manager  or  transportation  provider.  Government  and 
the  relocation  management  stakeholders  place  the  highest  value  on  expanding 
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move  management  services.  Servicemembers  and  transportation  providers  feel 
that  as  a  whole  these  services  are  moderately  important.  A  single  point  of  contact 
during  the  move  process,  shipment  tracking,  and  delivery  coordination  are 
perceived  as  providing  the  most  value  an  aggregate  of  stakeholders.  While  real 
estate  assistance  is  seen  as  critical  to  improving  customer  satisfaction  by  the 
relocation  industry,  the  majority  of  other  stakeholders  place  very  little  value  on 
this  criterion. 


2.  Reduce  Total  Cost  of  the  Personal  Property  Program 


Reduce  Total  Cost 

Stakeholder 

Average 

Std.T)ev. 

Aggregate 

3.31 

0.59 

Government 

3.47 

0.84 

Providers 

3.11 

0.45 

Relocation 

3.44 

1.13 

Industry 

3.28 

0.62 

Servicemembers 

3.18 

0.37 

The  goal  of  reducing  the  total  cost  of  the  personal  property  program  is 
moderately  valued  by  the  stakeholders.  Stakeholders  place  the  greatest  value  on 
changing  the  traffic  allocation  process  and  encouraging  the  use  of  electronic  data 
interchange  within  the  personal  property  program.  Stakeholders  have  strong 
feelings  on  the  type  of  pricing  strategy  used  in  the  policy  alternatives.  However, 
most  stakeholders  were  ambivalent  about  changes  to  the  category  and  service 
structure.  Servicemembers  demonstrated  a  discernible  lack  of  interest  in  efforts 


supporting  the  reduction  in  total  cost. 


a. 


Price  Strategy 


Price  Strategy 

Stakeholder 

Avera.ge 

Std.  Dev. 

Aggregate 

2.67 

0.82 

Government 

2.66 

0.81 

Providers 

2.69 

1.03 

Relocation 

2.44 

1.80 

Industry 

2.58 

1.02 

Servicemembers 

2.76 

0.77 

Stakeholders  are  unanimous  in  their  opposition  to  a  continuation  of 
the  current  DoD  solicitation  process.  Moreover,  servicemembers  do  not  see 
pricing  strategies  as  their  concern.  The  majority  of  stakeholders  support  the  use  of 
a  domestic  commercial  tariff  and  an  international  single  factor  rate. 

Transportation  providers  support  the  use  of  a  commercial  tariff,  but  resist  efforts 
to  deeply  discount  the  commercial  rate.  Transportation  providers  oppose 
commissions  for  move  managers  as  a  violation  of  the  Anti-Kickback  Act.^ 
Government  stakeholders  and  move  managers,  on  the  other  hand,  believe 
commissions  are  representative  of  common  commercial  practices  and  should  be 
allowed  in  the  personal  property  program. 


®  The  Anti-Kickback  Act  defines  a  kickback  as  anything  of  value  provided  to  a  prime  contractor  for  the 
purpose  of  improperly  obtaining  favorable  treatment  in  connection  with  a  government  subcontract. 
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b. 


Traffic  Allocation 


Traffic  Allocation 

Stakeholder 

Aggregate 

3.59 

0.35 

Government 

3.64 

0.53 

Providers 

2.75 

1.24 

Relocation 

4.33 

0.80 

Industry 

3.54 

Servicemembers 

3.58 

0.36 

Stakeholders  place  a  high  value  on  using  quality  as  a  basis  for 
assigning  traffic  to  carriers.  However,  many  stakeholders  stated  that  traffic  should 
not  be  allocated  on  quality  alone,  but  should  be  based  on  “best  value.”  Best  value 
describes  a  process  for  allocating  traffic  based  primary  on  quality,  but  also 
considers  cost  a  factor.  The  transportation  industry  views  contract  consolidation 
and  a  reduction  in  contract  awards  as  anti-small  business.  Government 
stakeholders  and  relocation  managers  feel  these  two  criteria  are  significant  ways  to 
reduce  administrative  requirements  and  reduce  costs.  The  majority  of 
stakeholders  support  a  policy  in  which  either  a  move  manager  or  the 
servicemember  performs  traffic  distribution.  Transportation  providers  are  in 
opposition  to  any  policy  that  allows  move  managers  to  choose  carriers.  There  is  a 
consensus  amongst  stakeholders  on  increasing  the  length  of  contract  awards  to  at 
least  one  year. 


’  “Carriers”  include  both  motor  carriers  and  freight  forwarders 
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Electronic  Data  Interchange  (EDI)  Capability 


c. 


EDI  Capability 

Stakeholder 

Average 

Std.  Dev. 

Aggregate 

3.99 

Government 

4.57 

Providers 

3.50 

Relocation 

4.50 

Industry 

4.00 

Servicemembers 

3.40 

Government  and  industry  stakeholders  place  a  relatively  high  value 
on  encouraging  the  use  of  electronic  data  interchange  within  the  personal  property 
program.  Servicemembers  view  EDI  as  moderately  important  to  the  reengineering 
effort.  Government  believes  EDI  is  critical  to  achieving  future  cost  savings. 
Industry  stakeholders  see  EDI  is  a  viable  goal,  but  are  concerned  that  requiring  a 
level  of  EDI  capability  may  place  an  excessive  burden  on  small  businesses. 

d.  Reduce  Complexity  of  Movement  Category  and  Service 

Structure 


Category  &  Service  Structure 

Stakeholder 

Average 

Std.  Dev. 

Aggregate 

Government 

Providers 

3.50 

Relocation 

2.50 

Industry 

3.00 

Servicemembers 

3.00 

Stakeholders  place  only  a  moderate  value  on  reducing  the 

complexity  of  the  movement  category  and  service  structure.  Relatively,  most 

stakeholders  viewed  this  criterion  as  a  low  priority.  In  consonance  with  this 
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viewpoint,  the  policy  alternatives  minimize  changes  to  these  categories  and 
services. 


3.  Maintain  a  Competitive  Market 


Maintain  a  Competitive  Market 

Average 

Std.  Dev. 

Aggregate 

3.26 

0.38 

Government 

3.19 

0.36 

Providers 

4.03 

0.71 

Relocation 

3.44 

1.36 

Industry 

3.63 

0.76 

Servicemembers 

2.95 

0.43 

Government  and  industry  stakeholders  value  the  contributions  of 
small  businesses  to  the  personal  property  program  highly.  However,  there  are 
differing  opinions  on  the  role  small  business  should  play  in  the  program. 
Transportation  providers  place  a  high  value  on  limiting  the  impact  of  policy 
changes  on  small  business.  They  have  lobbied  congress  strongly  on  this  point. 
Government  stakeholders  and  move  managers  assert  that  current  laws  protect 
small  businesses  and  the  potential  impact  on  small  business  should  be  only 
moderately  valued.  Servicemembers  expressed  very  little  interest  in  this  goal. 
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a. 


Simplified  and  Reduced  Traffic  Structure 


Traffic  Structure 

Stakeholder 

Average 

Std.  D^. 

Aggregate 

3.69 

Government 

3.57 

Providers 

4.00 

Relocation 

5.00 

Industry 

4.50 

Servicemembers 

3.00 

The  general  agreement  amongst  stakeholders  is  that  the  traffic 
structure  should  be  simplified  and  reduced.  There  are  currently  over  17,000  traffic 
lanes  used  by  the  personal  property  program.  While  none  of  the  current  policy 
alternatives  emphasize  this  criterion,  each  alternative  has  attempted  to  streamline 
the  existing  network  in  some  manner. 

b.  Payment  Options 


Payment  Options 

Stakeholder 

Std:  Dev. 

Aggregate 

3.10 

0.86 

Government 

Providers 

3.33 

Relocation 

2.83 

1.61 

Industry 

3.08 

1.15 

Servicemembers 

3.36 

Many  stakeholders  feel  that  DFAS  should  be  the  best  option. 
However,  due  to  DFAS’  poor  performance  record,  most  recommend  using  a 
Government  Credit  Card  or  a  Bank  Service  Contract.  Government  stakeholders 
are  wary  of  the  significant  administrative  burden  caused  by  the  use  of  the 
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Government  Credit  Card.  Many  stakeholders  indicated  that  they  support  the  use 
of  a  bank  service  contract  not  because  it  provided  the  greatest  value,  but  because  it 
is  the  least  objectionable  option. 

c.  Small  Business  Incentives 


Small  Business  Incentives 

Stakeholder  .  , 

Average 

Std.Dev. 

Aggregate 

2.98 

0.73 

Government 

3-14 

Providers 

0.35 

Relocation 

2.50 

2.12 

Industry 

gggifl 

1.36 

Servicemembers 

There  was  a  large  range  of  values  placed  on  limiting  the  impact  of  a 
policy  change  on  small  businesses.  Government  and  the  relocation  industry  view 
small  business  as  a  critical  component  of  the  moving  industry’s  capacity. 
However,  they  feel  that  small  businesses  can  be  integrated  into  a  network  of 
carriers  and  freight  forwarders  to  achieve  greater  service  and  less  cost.  Many 
small  businesses  have  exclusive  relationships  with  national  carriers.  This  has 
caused  widespread  cynicism  among  government  stakeholders  when  denoting 
which  businesses  are  “small”  and  which  are  not.  Government  and  relocation 
industry  stakeholders  feel  that  meeting  small  business  requirements  under  the 
Small  Business  Act  are  sufficient  for  ensuring  small  business  participation  in  the 
personal  property  program.  Transportation  providers  disagree  with  this  view  and 
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support  the  introduction  of  policy  features  that  limit  or  dispel  the  authority  of  the 
move  manager  and  increase  the  participation  of  small  businesses  in  the  program. 

C.  CHAPTER  SUMMARY 

The  interview  process  requires  the  collection,  coding  and  reduction  of  the 
data  into  a  meaningful  framework.  Data  reduction  refers  to  the  process  of 
selecting,  focusing,  simplifying,  abstracting,  and  transforming  the  “raw”  data  that 
are  generated  from  the  interviews  (Roberts  and  King,  1996.  250).  Coding  data 
allows  for  the  interpretation  and  integration  of  the  results.  The  valuation  of  the 
evaluation  criteria  by  the  stakeholders  provides  a  basis  for  the  evaluation  of  the 
policy  alternatives  from  the  each  stakeholder  group’s  perspective.  Chapter  V, 
Data  Analysis,  will  evaluate  the  alternative  policies  for  reengineering  DoD’s 
personal  property  program  using  a  single  set  of  evaluation  criteria  derived  from 
the  stakeholder  analysis. 
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V.  DATA  ANALYSIS 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  evaluate  the  alternative  policies  for 
reengineering  DoD’s  personal  property  program  using  a  single  set  of  evaluation 
criteria  derived  from  a  stakeholder  analysis.  Figure  12  displays  the  steps  and 
supporting  tasks  that  are  used  to  analyze  the  data  summarized  in  Chapter  IV. 
Statistical  standards  and  methods  are  employed  to  compare  and  contrast  the  three 
alternatives.  The  result  of  this  process  is  the  identification  of  a  policy  alternative 
or  alternatives  that  best  satisfy  stakeholder  criteria. 


Data  Analysis  Overview 
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Chapter  V 

Appendix  D 
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criteria 

Chapter  V 
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13-18 

Compare  and  contrast  total 
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Figure  12.  Data  Analysis  Overview 


B.  EVALUATION  OF  POLICY  ALTERNATIVES 

The  policy  alternatives  were  evaluated  from  each  stakeholder  perspective  as 
well  as  an  aggregate  perspective  using  a  single  set  of  criteria.  Interview  data  were 
used  to  generate  values  for  the  evaluation  criteria.  The  arithmetic  mean  of 
individual  stakeholder  responses  represents  the  value  a  stakeholder  group  places 
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on  a  criterion.  I  then  made  a  comparison  between  the  mean  for  each  criterion  and 
the  value  assigned  to  a  policy  alternative  shown  in  Figure  9. 

The  absolute  value  of  the  difference  between  the  stakeholder  group’s 
expressed  values  and  the  value  I  assigned  to  each  policy  alternative  was  calculated 
from  this  comparison.  The  difference  between  these  two  values  indicates  the 
degree  to  which  a  policy  alternative  fulfills  stakeholder  e?q)ectations  for  that 
criterion.  The  lack  of  a  difference  indicates  that  the  stakeholder’s  expectations 
have  been  more  fully  met.  Thus,  the  greater  the  difference,  the  greater  the  degree 
of  incompatibility  between  the  stakeholder  values  and  the  policy  alternative. 

For  example,  government  stakeholders  placed  an  average  value  of  “3.33” 
on  the  use  of  FAR-based  contracts.  I  assigned  a  value  of  “5”  to  this  criterion  for 
the  MTMC  Pilot.  The  absolute  value  of  the  difference  between  the  stakeholder 
group’s  expectations  and  the  value  I  assigned  to  the  MTMC  pilot  is  “1.67  (5- 
3.33=1.67).”  The  SAM  pilot,  on  the  other  hand,  has  an  assigned  value  of  “3”  for 
this  criterion.  Therefore,  the  difference  in  this  case  would  be  “.33.”  Thus,  the 
SAM  pilot  comes  closer  to  meeting  government  stakeholder  expectations  for  the 
use  of  FAR-based  contracts. 

1.  Evaluation  of  Aggregate 

The  final  step  of  the  evaluation  process  is  to  determine  the  policy 
alternative  that  best  satisfies  the  stakeholder’s  criteria  as  a  whole.  This  was 
accomplished  by  totaling  the  differences  between  a  stakeholder  group’s 
expectations  and  a  policy  alternative’s  values.  The  total  difference  that  is  closest 
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to  zero  designates  the  policy  alternative  that  best  satisfies  stakeholder 
expectations.  Figure  13  presents  the  results  of  the  evaluation  for  the  aggregate  of 
the  stakeholder  groups.  Overall,  the  FSMP  best  satisfies  stakeholder  criteria.  This 
is  in  a  large  part  due  to  the  move  management  services  that  the  FSMP  provides  to 
the  servicemember.  The  FSMP  consistently  fulfills  stakeholder  values  to  a  greater 
extent  than  the  other  alternatives  for  the  majority  of  sub-goals.  Each  policy 
alternative  meets  stakeholder  expectations  for  reducing  total  cost  and  maintaining 


a  competitive  market  to  a  similar  extent.  The  largest  disparity  amongst 
alternatives  in  maintaining  a  competitive  market  is  for  payment  options.  The 
FSMP  proposes  the  use  of  a  bank  service  contract  as  a  payment  option.  The  SAM 
uses  a  government  credit  card,  and  the  MTMC  pilot  continues  to  use  DFAS. 
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2.  Evaluation  of  Government  Stakeholder  Group 

Figure  14  presents  the  results  of  the  evaluation  for  the  government 
stakeholder  group.  Overall,  the  FSMP  best  satisfies  government  stakeholder 
criteria.  The  values  government  stakeholders  place  on  the  sub-goals  supporting 
improving  the  quality  of  life  for  servicemembers  are  congruent  with  the  FSMP. 
However,  the  alternatives  are  nearly  equivalent  in  meeting  government 
stakeholder  values  for  loss  and  damage  claims.  The  MTMC  pilot  does  as  well  as 
the  FSMP  in  meeting  the  government’s  criteria  for  reducing  total  cost. 
Furthermore,  the  SAM  pilot  meets  the  government’s  criteria  for  maintaining  a 
competitive  market  to  a  similar  degree  as  the  FSMP.  However,  the  FSMP  comes 
closer  than  the  other  alternatives  to  meeting  government  expectations  for  both 
reducing  total  cost  and  maintaining  a  competitive  market. 
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Figure  14.  Government  Total  Difference 


3.  Evaluation  of  Transportation  Provider  Stakeholder  Sub-Group 

Figure  15  presents  the  results  of  the  evaluation  for  the  transportation 
provider  stakeholder  sub-group.  Altogether,  the  SAM  alternative  best  satisfies  the 
transportation  providers’  criteria.  This  is  due,  in  part,  to  the  extent  to  which  the 
SAM  pilot  fulfills  transportation  providers’  expectations  for  reducing  total  cost 
and  maintaining  a  competitive  market.  In  particular,  the  SAM  pilot  does  better 
than  the  other  two  alternatives  in  meeting  the  transportation  providers’  criteria  for 


allocating  traffic  and  supporting  small  businesses.  The  MTMC  pilot  alternative 


best  fulfills  the  values  transportation  providers  place  on  improving  customer 


satisfaction.  The  MTMC  pilot  does  well  in  fulfilling  stakeholder  criteria  for  loss 
and  damage  claims  and  provider  qualifications. 
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4.  Evaluation  of  Relocation  Manager  Stakeholder  Sub-Group 

Figure  16  presents  the  results  of  the  evaluation  for  the  relocation  manager 
stakeholder  sub-group.  The  FSMP  satisfies  the  relocation  managers’  criteria  by  a 
substantially  larger  margin  than  the  other  two  alternatives.  This  outcome  is 
consistent  with  the  similarities  between  the  relocation  management  concept  and 
the  focus  of  the  FSMP.  In  almost  every  regard,  the  values  relocation  managers 


place  on  the  criteria  are  comparable  to  the  values  I  assigned  to  the  FSMP. 
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Figure  16.  Relocation  Managers  Total  Difference 


5.  Evaluation  of  Industry  Stakeholder  Group 

Figure  17  presents  results  of  the  evaluation  for  the  industry  stakeholder 
group.  As  a  whole,  industry  stakeholder  criteria  are  best  satisfied  by  the  FSMP. 
The  FSMP  offers  the  greatest  congruity  with  stakeholder  expectations  for 
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improving  quality  of  life  for  the  servicemember.  There  is  a  degree  of  equality 
between  the  SAM,  MTMC  and  the  FSMP  when  evaluating  the  ability  of  these 
alternatives  to  meet  industry  stakeholders  desires  for  reducing  total  cost  and 


maintaining  a  competitive  market.  Industry  stakeholders  do  not  appear  to  support 


either  the  SAM  pilot’s  approach  to  provider  qualifications  or  the  MTMC  pilot’s 


use  of  DFAS  as  a  payment  option. 
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Figure  17.  Industry  Total  Difference 


6.  Evaluation  of  Servicemember  Stakeholder  Group 
Figure  18  presents  the  results  of  the  evaluation  for  the  servicemember 
stakeholder  group.  Overall,  servicemembers  demonstrate  a  preference  for  the 
FSMP  over  the  other  alternatives.  Servicemembers  prefer  the  MTMC  alternative 
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with  regards  to  claims  settlement.  However,  they  value  the  move  management 
services  provided  by  the  FSMP. 
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C.  CHAPTER  SUMMARY 

This  chapter  presents  the  data  analysis  necessary  to  support  the  evaluation 
of  the  three  alternatives  to  reengineer  the  DoD’s  personal  property  program  using 
a  stakeholder  approach.  The  policy  alternatives  were  evaluated  from  each 
stakeholder  perspective  as  well  as  an  aggregate  perspective  using  a  single  set  of 
criteria.  The  result  of  this  process  is  the  identification  of  the  Full  Service  Moving 
Program  as  the  alternative  that  best  satisfies  each  stakeholders  group’s  criteria. 
Chapter  VT  draws  conclusions  and  recommendations  based  on  this  analysis. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

The  Military  Traffic  Management  Command  (MTMC)  is  currently 
reengineering  the  DoD  Personal  Property  Shipment  and  Storage  Program.  DoD  is 
conducting  three  pilot  projects  in  support  of  this  effort.  Each  pilot  project 
represents  a  policy  alternative  for  improving  moving  services  for  military  families. 
This  thesis  evaluates  the  policy  alternatives  for  reengineering  the  DoD  personal 
property  program  using  a  stakeholder  approach. 

The  study  provides  a  model  for  evaluating  policy  alternatives.  Based  on 
this  model,  evaluation  criteria  were  established.  Values  were  determined  for  these 
criteria  by  interviewing  a  convenience  sample  of  stakeholders.  Policy  alternatives 
were  then  analyzed  fi'om  each  stakeholder  perspective.  Finally,  a  policy 
alternative  was  identified  that  best  satisfies  the  criteria  for  each  stakeholder  as  well 
as  the  aggregate  of  stakeholders.  This  chapter  will  summarize  the  study’s  findings 
based  on  the  original  research  questions. 

B.  RESTATEMENT  OF  RESEARCH  QUESTIONS 
Primary  Research  Question: 

Which  policy  alternative  for  reengineering  the  DoD’s  personal  property 
program  best  satisfies  stakeholder  criteria? 
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Subsidiary  Research  Questions: 

1 .  What  are  the  policy  alternatives  for  reengineering  the  personal 
property  program? 

2.  What  are  the  criteria  for  evaluating  for  the  policy  alternatives? 

3 .  Who  are  the  stakeholders  for  the  reengineering  effort? 

4.  What  value  do  the  stakeholders  place  on  the  evaluation  criteria? 

5.  Which  policy  alternative  fulfills  stakeholder  criteria? 

C.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  Full  Service  Move  Program  (FSMP)  best  satisfies  the  criteria  of  all 
three  stakeholders  groups  in  aggregate.  The  extent  to  which  the  FSMP  met 
stakeholders’  expectations  varied  from  stakeholder  to  stakeholder.  However,  the 
FSMP  met  stakeholder  values  for  improving  quality  of  life  for  the  servicemember 
to  the  greatest  extent. 

All  stakeholder  groups  place  the  highest  value  on  improving  the 
servicemember’s  quality  of  life.  Stakeholders  are  consistent  in  placing  the  highest 
values  on  the  following  criteria: 

•  Full  Replacement  Value  for  Personal  Property 

•  Reduced  Claim  Preparation  Requirements 

•  Reduced  Claim  Settlement  Period 

•  Providers’  Qualification  Based  on  Past  Performance 

•  Single  Point  of  Contact  for  the  Move  Process 

•  Improved  Shipment  Tracking 

•  Improved  Delivery  Coordination. 
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Stakeholders  place  the  lowest  value  on  reducing  the  impact  on  small 
business.  Analysis  of  the  interview  results  demonstrated  that  the  following  criteria 
are  the  most  divisive  or  lowest  valued: 

•  Claims  Settlement  with  the  Government 

•  Inclusion  of  the  Service  Contract  Act 

•  Real  Estate  Assistance 

•  Continued  use  of  the  Current  DoD  Solicitation  Process 

•  Distribution  of  Traffic  by  the  Government 

•  Payment  by  DFAS 

•  Small  Business  Prime  Contract  Set-Asides. 

During  the  course  of  the  interview  process,  stakeholders,  especially 
servicemembers,  consistently  placed  a  high  value  on  improving  the  quality  of  life 
for  servicemembers.  Moreover,  there  is  a  consensus  among  stakeholders  on  issues 
relating  to  providing  full  replacement  insurance  and  improving  the  claims  process. 
Additionally,  many  of  the  expanded  services  that  servicemembers  place  the 
highest  value  on  can  be  provided  under  the  current  system.  Servicemember 
satisfaction  can  be  substantially  improved  by  implementing  non-divisive  quality  of 
life  improvements.  The  most  divisive  issues  identified  in  this  study  are  mainly 
related  to  efforts  to  reduce  total  cost  and  maintain  a  competitive  market. 
Disagreement  between  stakeholders  on  aspects  of  these  two  goals  has  been  the 
primary  cause  of  delays  and  difficulties  in  developing  and  testing  alternative 
polices.  Specifically,  government’s  plan  to  use  a  relocation  management  company 
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to  manage  the  personal  property  program  has  been  vehemently  opposed  by 
transportation  providers. 

In  expanding  the  Hunter  Pilot  into  the  FSMP,  the  government  was  forced  to 
compromise  on  a  number  of  critical  components  of  this  alternative.  These 
compromises  have  diminished  the  impact  a  relocation  management  company  can 
have  on  the  personal  property  program.  In  its  proposed  version,  the  FSMP  does 
not  provide  the  relocation  management  company  with  the  authority  it  has  over 
carriers  in  a  purely  commercial  environment,  yet  continues  to  hold  the  relocation 
company  to  coihmercial  standards.  Thus,  the  commercial  move  concept  can  not 
be  adequately  tested  in  a  diluted  form.  A  short-term  solution  to  this  impasse,  may 
be  an  expansion  of  the  SAM  or  MTMC  pilots  or  the  development  of  a  separate 
pilot  to  test  the  feasibility  of  achieving  quality  of  life  goals  without  the  use  of  a 
relocation  management  company. 

D.  LIMITATIONS 

This  study  faced  a  number  of  limitations  that  are  common  to  policy 
analysis.  These  limitations  can  be  grouped  into  three  major  categories:  scope 
limitations,  institutional  obstacles,  and  technical  obstacles.  Each  of  these 
limitations  provides  a  challenge  to  the  evaluation  process. 

1.  Scope  Limitations 

The  scope  of  this  thesis  was  limited  to  an  evaluation  of  the  current  policy 
alternatives.  No  conscious  effort  was  made  to  identify  other  alternatives  or 
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solutions  to  the  presenting  problem.  Interviewees  did  provide  other  options  that 
did  not  fit  within  the  context  of  this  study.  Furthermore,  evaluation  criteria  were 
gauged  to  assess  the  differences  between  alternatives  and  determine  the  values  of 
stakeholders.  As  a  result,  the  criteria  are  not  useful  in  identifying  other  criteria 
that  may  be  hi^ly  valued  by  the  stakeholders. 

2.  Institutional  Obstacles 

Institutional  obstacles  include  the  influence  of  political  pressure  on  the 
policy  development  and  evaluation,  the  effect  of  multiple  competing  stakeholders 
on  the  policy  evaluation  process,  and  the  difficulties  in  assessing  a  complex  public 
policy  initiative.  The  direction  the  reengineering  effort  has  taken  has  been 
influenced  by  the  dynamics  of  the  political  environment.  Furthermore,  the 
complexity  and  size  of  the  personal  property  program  coupled  with  numerous 
stakeholders  makes  it  difficult  to  develop  criteria  capable  of  encompassing  all 
aspects  of  the  reengineering  effort. 

The  policy  evaluation  effort  faces  significant  obstacles  unique  to  the  public 
sector.  Political  maneuvering  can  often  cause  one  policy  alternative  to  be  favored 
at  the  expense  of  another.  Furthermore,  restrictions  may  be  placed  on  the 
evaluation  process  that  limit  the  full  range  of  options  or  solutions  available  to  the 
decision-makers.  The  reengineering  effort  is  not  exempt  from  these  political 
dynamics.  DoD  has  tended  to  emphasize  the  FSMP  at  the  expense  of  the  other 
pilots,  especially  the  SAM  alternative.  DoD  has  also  concentrated  on  developing 


81 


an  alternative  that  is  palatable  to  all  stakeholders.  As  a  result,  pilot  projects  have 
been  used  to  determine  which  aspects  of  the  program  can  be  implemented 
successfully,  rather  than  a  means  to  evaluate  three  exclusive  alternatives  in  an 
environment  that  encourage  innovation  and  learning  from  mistakes. 

Compounding  this  problem  is  the  vagueness  of  stakeholders  in  articulating 
their  sub-goals  and  goals.  A  critical  step  in  the  evaluation  process  is  developing 
adequate  evaluation  criteria.  However,  the  presence  of  so  many  powerful 
stakeholder  groups  forced  the  evaluator  to  make  subjective  judgements  as  to 
which  goals  and  sub-goal  to  recognize  and  at  what  level  in  the  mapping  process  to 
place  them.  Furthermore,  the  complexity  of  the  personal  property  made  it  difficult 
to  discern  and  capture  the  nuances  embodied  in  certain  criteria.  Bonser, 
McGregor,  and  Oster  point  out  that  “the  real  world  of  public  policy  is  messy,  and 
it  is  not  usually  possible  to  assemble  all  of  the  information  that  one  would  like  in 
order  to  prepare  a  complete  assessment  of  all  conceivable  policy  options.  Most 
inquires  are  under  time  and  financial  pressures  and  suffer  from  incomplete  or 
inadequate  data  (Bonser,  McGregor,  and  Oster,  p.  50).”  A  concerted  effort  was 
made  to  validate  assumptions  and  probe  complex  issues  associated  with  the 
reengineering  effort  throu^out  the  evaluation  and  data  collection  process. 

3.  Technical  Obstacles 

Technical  obstacles  presented  themselves  in  the  form  of  time  and  resource 
constraints.  The  use  of  a  convenience  sample  provides  a  means  to  identify  the 


82 


stakeholder  values  relating  to  the  reengineering  effort.  However,  a  convenience 
sample  is  not  a  random  sample.  The  use  of  a  non-random  small  sample  introduces 
the  possibility  of  a  bias  in  any  results.  Thus,  the  results  cannot  be  generalized  to  a 
large  population. 

Additionally,  criteria  are  equally  weighted  for  the  purposes  of  this  study. 
The  values  of  the  stakeholders  were  used  to  choose  between  the  policy  alternatives 
on  a  criterion  by  criterion  basis.  There  was  no  attempt  to  weight  criteria 
differently  in  calculating  the  extent  each  policy  alternative  fulfilled  stakeholder 
expectations  as  a  whole. 

E.  GENERAL  COMMENTS 

During  the  course  of  the  interview  process,  various  respondents  often 
provided  additional  comments.  Stakeholder  comments  tended  to  focus  on  issues 
that  are  important  or  of  concern  to  them  personally.  Topics  discussed  here  are 
representative  of  the  comments  received  from  individual  stakeholders. 

A  number  of  comments  referred  to  changes  to  the  damage  claim  process. 
One  issue  is  the  lack  of  true  full  replacement  protection  for  the  servicemember  in 
any  of  the  proposals.  Each  proposal  increases  the  liability  of  the  carrier,  but  this 
liability  is  based  on  a  dollar  per  pound  formula.  There  was  also  concern  raised 
over  the  need  for  an  arbitration  mechanism  when  the  claim  is  settled  with  a  carrier. 

Another  concern  was  that  by  requiring  binding  estimates,  there  is  an 
incentive  for  transportation  providers  to  overcharge  servicemembers.  Carriers  are 
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compelled  to  inflate  estimates  in  order  to  reduce  the  risk  of  underestimating  the 
true  cost  of  the  additional  service.  Anecdotal  evidence  suggests  that  carriers  do 
not  lower  billing  charges  if  true  cost  is  less  than  the  estimate.  Efforts  to  limit 
servicemember  liability  may  prove  more  successful  by  managing  the  process,  and 
not  by  mandating  the  result. 

Finally,  numerous  “horror  stories”  were  told  by  servicemembers  during  the 
interviews.  A  central  theme  to  these  stories  is  the  frustration  servicemembers  have 
with  the  current  system.  Servicemember  often  feel  “powerless”  during  the  move 
process.  A  low  quality  move  process  is  often  coupled  with  poor  customer  service 
and  ineffective  feedback  to  the  servicemember  on  complaints.  Servicemembers 
are  unable  to  ensure  they  receive  the  best  carrier  and  when  they  receive  poor 
service  they  feel  no  one  is  held  responsible. 

F.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

This  study  focused  on  evaluating  the  policy  alternatives  for  the 
reengineering  of  the  DoD  Personal  Property  Shipment  and  Storage  Program  from 
a  stakeholder  approaeh.  This  study  did  not  seek  to  e?q)lore  the  dynamics  of  the 
developing  public  policy,  nor  did  it  attempt  to  address  implementation  issues. 
Further  research  would  be  useful  in  determining  critical  steps  and  possible 
obstacles  in  the  development  and  implementation  processes. 

The  major  contribution  of  this  study  is  the  development  of  a  model  to 
assess  stakeholder  views  and  values  on  alternative  policies.  Two  additional  steps 
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are  needed  to  generate  results  that  will  sustain  a  scientific  model.  Further  studies 
should  attempt  to  incorporate  these  two  steps.  The  reliability  of  this  study  can  be 
greatly  increased  by  an  assessment  of  the  pilot  projects  by  another  evaluator  or 
evaluators.  Furthermore,  the  use  of  a  random  sample  will  allow  the  results  of  the 
analysis  to  be  used  to  make  inferences  about  the  characteristics  of  a  population. 
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APPENDIX  A.  DEFINITIONS 


The  following  terms  have  been  defined  as  listed  below  for  the  purposes  of  this 
study: 

AMSA:  The  American  Moving  and  Storage  Association  is  a  national  trade 
association  of  the  moving  and  storage  industry. 

Best  Value:  A  procurement  strategy  that  takes  into  account  both  cost  and  quality 
in  awarding  contracts. 

CINC:  Commander  in  Chief  refers  to  the  commander  of  a  Unified  Command. 
Criterion:  A  performance  measure. 

DOD:  Department  of  Defense. 

DFAS:  The  Defense  Finance  and  Accounting  Service  provides  responsive, 
professional  finance  and  accounting  services  to  the  Department  of  Defense. 

EWG:  Executive  Working  Group 

FSMP:  The  Full  Service  Move  Program  is  an  expansion  of  the  Hunter  Pilot  with 
certain  modifications  based  on  earlier  lessons  learned  firom  the  Hunter  Test. 

GAO:  The  General  Accounting  Office  is  the  investigative  arm  of  Congress.  GAO 
is  charged  with  examining  matters  relating  to  the  receipt  and  disbursement  of 
public  funds.  GAO  performs  audits  and  evaluations  of  government  programs  and 
activities. 

Goal:  A  target  of  an  activity  or  policy  expressed  as  a  measurable  outcome. 

HGFA:  The  Household  Goods  Forwarders  Association  is  a  national  trade 
association  that  comprises  all  facets  of  the  moving  industry,  including  moving  and 
storage  firms,  national  van  lines,  line-haul  firms,  international  and  domestic 
forwarders,  port  agents,  claims  adjusters,  material  providers,  insurance  providers, 
technology  firms,  and  others. 

Hunter  Pilot:  Managed  by  the  Department  of  the  Army  as  a  separate  pilot  effort 
to  determine  whether  the  commercial  business  practice  of  outsourcing  can 
alleviate  known  problem  areas  in  the  personal  property  program. 
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Industry:  Includes  both  transportation  providers  and  relocation  managers. 

IPPT:  Integrated  Product  Process  Team. 

Military  Mobility  Coalition:  Is  an  ad  hoc  organization  formed  in  the  spring  of 
1996  to  represent  the  relocation  management  industry's  interests  in  working  with 
the  Department  of  Defense's  household  goods  reengineering  initiatives. 

MTMC:  The  Military  Traffic  Management  Command  is  a  sub-command  of  the 
United  States  Transportation  Command  that  provides  the  Department  of  Defense 
worldwide  single  port  management,  transportation,  and  traffic  management 
services;  deployment  planning  and  engineering. 

MTMC  Pilot:  Managed  by  the  Military  Traffic  Management  Command  in  an 
effort  to  correct  deficiencies  in  the  personal  property  program  through 
reengineering; 

OSD:  The  Office  of  the  Secretary  of  Defense  is  the  organization  that  establishes 
and  reviews  policies,  and  coordinates  activities  for  the  Secretary  of  Defense. 

Personal  Property  Shipment  and  Storage  Program:  The  Department  of 
Defense  (DOD)  program  that  is  managed  by  the  Military  Traffic  Management 
Command.  Provides  for  the  shipping  and  storing  of  DOD  military  and  civilian 
employees'  household  goods  and  personal  effects. 

Policy  Analysis:  A  systematic  approach  to  making  policy  choices.  Its  purpose  is 
to  assist  policymakers  in  choosing  a  preferred  course  of  action  amid  multiple 
complex  alternatives. 

Policy  Evaluation:  Emphasizes  examining  alternative  policies,  as  contrasted  to 
describing  them  or  explaining  why  they  exist. 

PPSO:  Personal  Property  Shipping  Offices  manage  the  daily  operations  of  the 
personal  property  program. 

Relocation  Managers:  Provide  a  wide  range  of  relocation  services  through  an 
integrated  network  of  transportation  providers,  real  estate  agents,  and  move 
coordinators. 
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SAM:  The  Servicemember  Arranged  Move  is  managed  by  the  Department  of 
Navy  as  a  separate  pilot  effort  to  test  the  option  of  allowing  servicemembers  to 
choose  their  own  carrier  as  well  as  coordinate  their  move  with  that  carrier. 
SECDEF:  Secretary  of  Defense. 

Stake:  A  claim  that  each  stakeholder  has  on  the  organization  or  program.  A  stake 
may  be  tangible  or  intangible,  explicit  or  implicit.  A  stake  may  also  be  political, 
economic,  social  or  psychological  in  nature. 

Stakeholder:  Those  parties  who  either  affect  or  are  affected  by  an  organization's 
actions,  behavior,  and  policies 

Sub-goal:  A  set  of  activities  that  support  a  goal. 

Transportation  Providers:  Includes  both  motor  carriers  and  freight  forwarders. 

TMO:  Traffic  Management  Offices  are  base  activities  that  arrange  the  movement 
of  household  goods  by  coordinating  with  the  commercial  carrier  industry  and  the 
military  customer. 

USTRANSCOM:  The  United  States  Transportation  Command  is  responsible  for 
planning  and  coordinating  strategic  mobility  for  the  United  States  Armed  Forces. 
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APPENDIX  B.  QUESTIONNAIRE 


Name:  Stakeholder  Group:  G  I  S  Sub-Group: 


Sub-goals 

Measurement  Criteria  and  Sub-criteria 

Extent 

Valued 

Contracting  Method 

FAR  -based 

Loss  and  Damage  Claim 

Full  Replacement 

Direct  Claim  Settlement 

Move  Manager 

Transportation  Provider 

Government 

Reduced  claim  preparation  requirements 

Reduced  claim  settlement  period 

Provider  Qualification 

Financial  stability 

Ensured  capacity 

Past  performance 

Inclusion  of  Service  Contract  Act 

Inconvenience  payments 

Provider  Performance 
Measurement 

Customer  Surveys 

Random  Inspections 

Periodic  Performance  Assessment 

Move  Management 

Pre-move  counseling 

Single  point  of  contact  move  management 

Binding  estimates  for  excess  costs 

Destination  information 

Shipment  tracking 

Delivery  coordination 

Real  Estate  assistance 

Pricing  Strategy 

Current  DOD  solicitation  process 

Domestic 

International 

Commercial  Tariff 

Domestic 

GSA  Tender  1-W 

Domestic 

International 

Single-F  actor-Rate 

Domestic 

International 

Commissions  for  move  manager 

Traffic  Allocation 

(Consolidation)  Fewer  contracts  providing  more  services  per  contract 

Traffic  distribution  based  solely  on  quality 

Traffic  distribution  performed  by: 

Move  Manager 

Servicemember 

Government 

Elimination  of  the  Traffic  Distribution  Record 

Reduction  in  Contracts  awarded 

Increase  length  of  Contract  award 

EDI  capability  required 

Category  and  Service 
Structure 

Reduction  in  number  of  movement  and  storage  categories 

Traffic  Structure 

Change  current  structure  of  AOR  to  RA 

Payment 

DFAS 

Bank  Service  Contract 

Government  Credit  Card 

SBA  Compliance  Means 

Application  of  Small  Business  Act 

Small  Business  prime  contractor  set  asides 
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My  goal  is  determine  the  value  you  place  on  a  number  of  criteria  relating  to  the 
reengineering  of  die  personal  property  program  (moving  and  storage  of  household 
goods).  I  will  name  an  objective  and  I  would  like  to  know  how  important  you  feel 
that  objective  is  to  any  effort  to  change  the  current  personal  property  program. 

Please  rate  the  objectives  using  the  following  scale: 

0  =  Not  applicable 

1  =  Not  Important 

2  =  Low  Importance 

3  =  Medium  Importance 

4  =  High  Importance 

5  =  Critical 
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APPENDIX  C.  INTERVIEW  RESPONSES 
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(2  government) 


(3  industry) 
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(4  servicemembers) 


Relocation 


(5  comparison  chart) 


Government  credit  card 


(6  goals  and  subgoals) 
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APPENDIX  D.  ANALYSIS 
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(2  government) 


Contracting  Method _ FAR  -based 

Loss  and  Damage  Claim  Full  Replacement 


(3  transportation  providers) 
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Market _ SBA  Compliance  Means _ Application  of  Small  Business  Act _ 

_ Small  Business  prime  contractor  set  asides 


Contracting  Method _ FAR  -based _ 

Loss  and  Damage  Claim _ Full  Replacement 

Direct  Claim  Settlement 


(5  industry) 
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SBA  Compliance  Means _ Application  of  Small  Business  Act _ _ 4.25  4 _ 0,25 _ 4 _ 0,25 _ 4 _ 0,25 

_ Small  Business  prime  contractor  set  asides _ 2.33  5 _ 2.67 _ 1  1.33  1  1.33 


(6  servicemember) 
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a  Competitive  _ Government  Credit  Card _ 4.00 _ 5 _ 100 _ 1 _ 3.00 _ 1 _ 3.00 

Market  SBA  Compliance  Means _ Application  of  Small  Business  Act _ 2.67 _ 4 _ 133 _ 4 _ 133 _ 4 _ 133 

Small  Business  prime  contractor  set  asides _ 2.33  5  2.67  1 _ 133  1  133 


(7  analysis) 
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